The FBI main headquarters are located in Washington, D.C. and is located in the J. Edgar Hoover Building.
The FBI has other offices in other states and towns but the main headquarters and FBI office is located in Washington D.C.
The FBI is the Federal Investigation Authority and does more investigation and an FBI agent has more arrest powers and more powers than a regular police officer does have.
So when police need more help in investigating something the FBI will take over and help them investigate especially federal crimes.
If someone does something that federal illegal then the FBI will take over on the case even though the police arrest the person.