Ralph Lauren’s decision to vacate office space at 625 Madison Ave. could lead to SL Green losing control of the building.

Ralph Lauren is consolidating offices and expanding to 350K SF at RXR Realty’s Starrett-Lehigh Building. The move means SL Green will have 300K SF vacant at 625 Madison, a 17-story Plaza District office building on which SL Green holds the ground lease, next year, Crain’s New York Business reports.

The ground rent at 625 Madison Ave. is $4.6M annually right now, but Ashkenazy Acquisition Corp. — which owns the property — has said it plans to sharply increase rent in order to force out SL Green.

An Ashkenazy executive told Crain’s two years ago that the rent should be closer to $80M. The lease expires in 2022, and if Ashkenazy were to take back the building, the outstanding leases with current tenants would no longer apply. That could make it hard for SL Green to lure new tenants and keep the existing ones from leaving over the looming uncertainty.

"There's no question it's going to be tricky to attract tenants to the building," Colliers International New York-area President Michael Cohen told Crain’s. "I would bring a tenant to the property, but I would also caution them that there are issues that may impede their ability to sign a long-term lease."

SL Green said that it is confident it will be able to fill the space left behind from the departure of Ralph Lauren, and that it will reach a suitable arrangement with the property owner.

"Ground leases are a normal part of doing business in New York City, and we expect this revaluation to be no different," SL Green Executive Vice President Steven Durels said.