Several bomb squads started combing the area shortly after employees were notified about the threat around 11:45 a.m. local time, according to an IBM Malaysia representative.
Other tenants of IBM's 23-story building include IT companies such as VADS, Silver Lake and Global Impact.
"The building is still off-limits now," the representative said, adding that the total number of workers affected could not yet be determined.
Meanwhile, about 2,000 people were evacuated from the Petronas Twin Towers after a similar bomb threat Wednesday morning. Workers and tourists milled around the complex as scores of ambulances and police personnel went on guard.
Multinational companies that lease offices at the Towers include Lucent Technologies, Reuters and consultancy company McKinsey & Co.
Some tenants were unaware of the bomb scare.
"We were told there was a fire drill and we had to walk down to the ground floor," said Foong Pek-Foong, with the information-security company Biodata.
Foong said some workers had collapsed from exhaustion after evacuating, as the elevator was inoperable. The building is 88 stories high.
On Tuesday, terrorists crashed two hijacked airliners into the World Trade Center in New York and brought down the twin 110-story towers. A jetliner also slammed into the Pentagon outside Washington.
A fourth jetliner, also apparently hijacked, crashed in Pennsylvania. U.S. President George W. Bush has ordered a full-scale investigation to "hunt down the folks who committed this act."
Thousands could be dead or injured, a high-ranking New York police official told AFP, speaking on condition of anonymity.
Daniel Lewin, Akamai's co-founder and chief technical officer, was onboard American Airlines Flight 11 from Boston to Los Angeles when it was hijacked and subsequently crashed into the Trade Center.
All U.S. exchanges and markets will remain closed Wednesday. Across Asia, stock exchanges in Taiwan and Malaysia have suspended trading.
Staff writer Sreejit Pillai reported from Malaysia.