Jermaine was married to Hazel Gordy when the rumoured affair began in 1984.
The 56-year-old singer, who has never spoken about their alleged affair, was so full of grief following Whitney's death last month he was unable to attend her funeral, The Sun on Sunday reports.
[Related article: Whitney Houston's funeral]
[Related feature: Whitney Houston tribute photo gallery]
Whitney's 1985 song 'Saving All My Love For You,' about a woman involved with a married man, mirrored her rumoured relationship with Jermaine. The video even featured a Jermaine lookalike.
Jermaine's brother, Michael Jackson is said to have been among the few people who knew about their relationship, and reportedly did not approve of their affair.
Following the end of their relationship, the pair stayed friends and kept in contact. Whitney comforted Jermaine after Michael's death in 2009.
[Related article: BAFTA stars pay tribute to Whitney Houston]
[Related article: Jennifer Hudson leads Grammys tributes to Whitney]
Jermaine wrote about Whitney in his 2011 book, You are Not Alone: Michael, Through A Brother's Eyes.
Although he does not write about their alleged affair, he does admit his feelings for her, writing: "As much as I wanted to lose myself in all these feelings, I told her to wait. Ultimately, we had to go our separate ways and it killed us both."
Writing about Michael's attitude, Jermaine added: "He didn't fuel the temptation as some guys would. He reminded me about Hazel, about family."
Jermaine and Hazel broke up three years after his alleged affair with Whitney.