Cage the Elephant lead singer Matt Schultz was arrested in New York City on Thursday and charged with criminal possession of a weapon, the New York Police Department confirmed to PEOPLE.
An employee at the Bowery Hotel on Manhattan's Lower East Side called the cops after they allegedly witnessed Schultz, 39, carrying a handgun into a public restroom on the ground floor on Wednesday night, according to the Daily Mail.
Police say they responded a day later and knocked on Schultz's door. He reportedly admitted to cops to owning two .45 caliber guns, but when asked by the police if he still had the weapons, he responded that he did not know, per the publication.
The NYPD says he was then arrested and taken to the 9th precinct station house.
A judge granted police a search warrant to enter his room where cops say they found two loaded guns, a Sig Sauer and a Smith & Wesson, according to the Daily Mail.
There are conflicting media reports about whether or not Schultz possessed a firearm license for the guns. The Daily Mail reported that Schultz did not have one, while ET Canada claims he did.
Schultz's reps have not yet responded to PEOPLE's request for comment.
The "Ain't No Rest for the Wicked" singer had reportedly left the 9th precinct station house briefly on Friday to pick up medication from Bellevue Hospital before returning a few hours later.
Schultz did not have any prior criminal history before his arrest on Thursday.
Rock band Cage the Elephant was formed in 2006 in Bowling Green, Kentucky. Schultz previously worked at a sandwich bar and as a plumber before joining the band.
The band found commercial and critical success as their sophomore album Thank You Happy Birthday debuted at No. 2 on the Billboard 200 and their 2015 studio album, Tell Me I'm Pretty, earned a Grammy award for best rock album. They also won another Grammy for best rock album for their 2019 album Social Cues.
The band is scheduled to perform in Toronto, Canada on Aug. 2 and at the Festivent in Quebec, Canada two days later. It is unknown yet if Schultz's arrest will affect these tour dates.
