Assistance to Firefighters Act of 2004 - Amends the Federal Fire Protection and Control Act of 1974 to authorize the Secretary of Homeland Security (currently, the Director of the Federal Emergency Management Agency (FEMA)) to make grants to State fire departments for the purpose of protecting the health and safety of the public and fire-fighting personnel against fire and fire-related hazards, and to provide assistance for fire prevention programs. Authorizes grants to: (1) volunteer emergency medical service (EMS) organizations; and (2) obtain automated external defibrillator devices.

Directs the Secretary to: (1) establish specific criteria for the preliminary review of applications; (2) review such criteria at least annually; (3) award grants based on the review of grant applications by a panel of fire service personnel appointed by a national organization recognized for expertise in the operation and administration of fire services; (4) provide for the administration of the review panel and ensure that an individual appointed to such panel is a recognized expert in fire-fighting, medical services provided by fire services, fire prevention, or research on firefighter safety; and (5) provide assistance with the preparation of grant applications.

Expands program eligibility. Reduces limitations regarding matching funds. Increases the amount a grant recipient may receive. Earmarks funds for volunteer EMS organizations.

Directs: (1) the Secretary to study and report to Congress on firefighter assistance; and (2) the Comptroller General to report to Congress on the administration of assistance and on the Secretary's success in administering FEMA.