Jennifer Hudson led the tributes as she performed an a cappella version of Whitney's classic 'I Will Always Love You.'
[Related feature: BAFTA stars pay tribute to Whitney Houston]
The 30-year-old sang under a TV screen, which had shown a video montage of music greats who died in the last year, including Amy Winehouse, Heavy D, Etta James and Whitney.
When she finished, Jennifer broke down in tears as the audience showed their appreciation.
Many at the ceremony paid tribute to Whitney, who was found dead in her hotel room on Saturday.
[Related feature: Whitney Houston tribute photo gallery]
Host of the show LL Cool J led the audience in a prayer of thanks, saying: "Heavenly Father, we thank you for sharing our sister Whitney with us.
"Though she is gone too soon, we remain truly blessed to have been touched by her beautiful spirit."
Kelly Rowland told E Entertainment: "We lost our hero. I would not be standing here today if it was not for her. My prayers go to her mother and her daughter. If we feel this loss I can only imagine how heavy their loss is."
[Related feature: Grammys 2012 red carpet fashion]
Kelly Osbourne told Ryan Seacrest: "I knew her personally and she really was a lovely woman."
The last person to perform with Whitney, Kelly Price broke down on the red carpet as she spoke about seeing star for the last time on Thursday night.
She said: "I was having a pre-Grammy party for the nominations. I invited Whitney to come.
"We had a great time Thursday night. She came out. We hung out. We danced. I wasn't expecting her to sing. She just walked up on the stage. We embraced each other and she said: 'Give me the mike.'"
[Related article: Grammys 2012 Adele scoops six awards]
Billy Ray Cyrus told Ryan: "I think there is a feeling of honour to respect and honour an incredible talent that this world has lost.
"As a human being she's always been very nice to me and my family when we've crossed paths throughout the years. She sang the backdrop of our lives."
Mitch and Janis Winehouse also paid tribute to Whitney when they accepted the award for best pop duo/group performance on behalf of their late daughter Amy Winehouse for her duet with Tony Bennett.
Mitch said: "Long live Whitney Houston. Long live Amy Winehouse and long live Etta James. There's a beautiful girl band up there."
Grammy Awards 2012 winners
Record of the Year — Rolling In The Deep, Adele
Album of the Year — 21, Adele
Song of the Year — Rolling In The Deep, Adele
Best New Artist — Bon Iver
Best Pop Solo Performance —Someone Like You, Adele
Best Pop Duo/Group Performance — Body And Soul , Tony Bennett & Amy Winehouse
Best Pop Vocal Album — 21, Adele
Best Dance Recording - Scary Monsters and Nice Sprites, Skrillex
Best Dance/ Electronica Album - Scary Monsters and Nice Sprites, Skrillex
Best Traditional Pop Vocal Album — Duets II, Tony Bennett & Various Artists
Best Rock Performance — Walk, Foo Fighters
Best Rock Song — Walk, Foo Fighters
Best Rock Album - Waiting Light, Foo Fighters
Best R&B Performance — Is This Love, Corinne Bailey Rae
Best R&B Song — Fool For You
Best R&B Album — F.A.M.E., Chris Brown
Best Rap Performance — Otis, Jay-Z and Kanye West
Best Country Solo Performance — Mean, Taylor Swift
Best Country Song — Mean, Taylor Swift
Best Short Form Music Video — Adele