Maroochy Sunshine was contractually obliged to provide 22 workers from Vanuatu, who were in Australia as part of the Seasonal Worker Program, $500 a week for at least 30 hours' work.
The company allegedly arranged sporadic work in the Lockyer Valley, Sunshine Coast and Bundaberg areas in July 2014.
Thirteen workers were paid nothing for their work, the Fair Work Ombudsman claimed.
Other workers were allegedly paid amounts between $50 and $300, were underpaid on their leave entitlements and received their pay infrequently.
In total, the workers were allegedly underpaid $77,649.
Fairfax Media offered the company an opportunity to respond to the allegations but had not received a response.
The Fair Work Ombudsman took legal action against Emmanuel Bani and his company Maroochy Sunshine over the allegations, after receiving a referral from the Department of Employment.
Fair Work Ombudsman Natalie James said information provided by the workers to inspectors raised concerns.
"The allegations made by the workers are highly concerning and we have chosen to commence legal action because of the seriousness of the alleged contraventions," James said.
James said Bani had previously been put on notice of the need to pay employees' minimum lawful entitlements.
Fair Work inspectors allegedly met with Bani in relation to another matter, five days before the 22 workers from Vanuatu arrived, and reminded him of his obligations to comply with workplace laws.
James said it was important that the integrity of the Commonwealth Seasonal Worker Program be upheld.
"The Seasonal Worker Program helps contribute to the economic development of participating countries, while also offering Australian employers in the horticulture industry the ability to employ workers from selected Pacific Island countries and Timor-Leste when they cannot find enough local labour to satisfy seasonal demand," she said.
"People from the Pacific region and Timor-Leste can work in Australia on a short-term basis under the program, which is open to employers in agricultural industries, including the horticulture, aquaculture, cane sugar and cotton growing industries.
"It is also open to employers in certain locations across Australia in accommodation industries."
Bani faces penalties of up to $10,200 for each contravention and his company faces penalties of up to $51,000 for each contravention.
A directions hearing is scheduled for January 18 in the Federal Circuit Court in Brisbane.