McRae was born in Harlem to Jamaican immigrant parents, Osmond and Evadne McRae. She began studying piano when she was eight, and the music of jazz greats like Louis Armstrong and Duke Ellington filled her home. She met singer Billie Holiday when she was just 17 years old. As a teenager McRae came to the attention of Teddy Wilson and his wife, the composer Irene Kitchings Wilson. One of McRae's early songs, "Dream of Life" through their influence, was recorded in 1939 by Wilson’s longtime collaborator Billie Holiday. McRae considered Holiday to be her primary influence.
In her late teens and early twenties, McRae played piano at a New York club called Minton's Playhouse, Harlem's most famous jazz club, sang as a chorus girl, and worked as a secretary. It was at Minton's where she met trumpeter Dizzy Gillespie, bassist Oscar Pettiford, and drummer Kenny Clarke. Had her first important job as a pianist with the Benny Carter's big band (1944), worked with Count Basie (1944) and made first recording as pianist with Mercer Ellington Band (1946-1947). But it was while working in Brooklyn that she came to the attention of Decca’s Milt Gabler. Her five year association with Decca yielded 12 LPs.
In 1948 she moved to Chicago with comedian George Kirby. She played piano steadily for almost four years before returning to New York. Those years in Chicago, McRae told Jazz Forum, "gave me whatever it is that I have now. That's the most prominent schooling I ever had."Back in New York in the early 1950s, McRae got the record contract that launched her career. In 1954, she was voted best new female vocalist by Down Beat magazine. She married bassist Ike Isaacs in the late 1950s.
Among her most interesting recording projects were Mad About The Man (1957) with composer Noël Coward, Boy Meets Girl (1957) with Sammy Davis, Jr., participating in Dave Brubeck's The Real Ambassadors (1961) with Louis Armstrong, a tribute album You're Lookin' at Me (A Collection of Nat King Cole Songs) (1983), cutting an album of live duets with Betty Carter, The Carmen McRae-Betty Carter Duets (1987), being accompanied by Dave Brubeck and George Shearing, and closing her career with brilliant tributes to Thelonious Monk, Carmen Sings Monk (1990), and Sarah Vaughan, Sarah: Dedicated to You (1991).
As a result of her early friendship with Billie Holiday, she never performed without singing at least one song associated with "Lady Day", and recorded an album in 1983 in her honor entitled For Lady Day, which was released in 1995. Some songs included are; "Good Morning Heartache", "Them There Eyes", "Lover Man", "God Bless the Child", "Don't Explain", just to name a few. McRae also recorded with the world best jazz musicians, Take Five Live (1961) with Dave Brubeck, Heat Wave (1982) with Cal Tjader, and Two for the Road (1989) with George Shearing.
Carmen McRae sang in jazz clubs throughout the United States—and across the world—for over fifty years. McRae was a popular performer at the legendary Monterey Jazz Festival (1961-1963, 1966, 1971, 1973, 1982). Performing with Duke Ellington's at the North Sea Jazz Festival in 1980, singing "Don't Get Around Much Any More", and at the Montreux Jazz Festival in 1989.