Temporary Agricultural Work Reform Act of 2004 - Amends the Immigration and Nationality Act to revise the H-2A (temporary agricultural worker) program.

Eliminates the labor certification process and instead requires employers seeking H-2A workers to provide attestations concerning: (1) wages and working conditions; (2) the nondisplacement of U.S. workers; (3) the lack of a strike or lockout; and (4) compliance with prior H-2A approvals. Establishes penalties for failing to meet, or misrepresentations concerning, attestations.

Replaces the adverse effect wage rate with a prevailing wage rate. Requires employers of H-2A workers to: (1) pay the greater of the prevailing wage or the applicable State minimum wage; (2) offer to provide housing or a housing allowance; (3) reimburse costs of initial transportation to the place of employment and provide transportation to the work site; and (4) guarantee employment for a specified period of time.

Requires the Secretary of Homeland Security to establish a procedure for the expedited adjudication of H-2A petitions.

Renders inadmissible and ineligible for H-2A status those aliens who previously violated the terms of their admission. Provides a one-time waiver for those who are outside the United States.

Amends the Immigration Reform and Control Act of 1986 to prohibit the Legal Services Corporation from: (1) providing legal assistance to aliens unless those aliens are present in the United States as H-2A workers; and (2) bringing a civil action on behalf of H-2A workers unless the Federal Mediation and Conciliation Service has attempted to mediate the dispute.