AI Governance for Mid-Sized Businesses: Where Do You Actually Start?
AI is already being used across Australian businesses.
Employees are using tools such as ChatGPT, Microsoft Copilot and other AI-enabled software to draft documents, summarise meetings, analyse information, create content, write code and perform everyday tasks.
At the same time, businesses are purchasing software that increasingly has AI built into it.
For many mid-sized businesses, the question is no longer whether the business will use AI. It is how to make sure AI is being used appropriately — without creating unnecessary legal, commercial or reputational risk.
This is where AI governance comes in.
But AI governance can sound much more complicated than it needs to be.
You do not necessarily need to start with a large governance framework or a 50-page AI policy.
A much better starting point is understanding where AI is being used, what risks that use creates, and what controls are actually needed for your business.
What does AI governance actually mean?
At its simplest, AI governance is the way your business decides:
what AI can be used for;
what AI should not be used for;
who is responsible for decisions about AI;
what information can be entered into AI systems;
how new AI tools are assessed before they are introduced;
when a person needs to check or approve an AI-generated output; and
how the business responds if something goes wrong.
It does not need to mean creating an entirely new governance structure.
In many businesses, AI governance can sit within existing processes for privacy, cybersecurity, procurement, risk, technology and compliance.
The Australian Government's current AI guidance takes a similar approach. It recommends establishing clear accountability for AI, assessing risks, protecting data, testing AI systems and maintaining appropriate human oversight.
The important point is that governance should reflect how your business actually uses AI.
A business using AI to help draft internal marketing copy does not necessarily require the same controls as a business using AI to assess job applicants, analyse customer information or make recommendations that affect customers.
Step 1: Find out where AI is already being used
Before preparing an AI policy, start by understanding what is actually happening within the business.
This can be surprisingly difficult.
AI may already be used through:
publicly available generative AI tools;
Microsoft 365 or other productivity software;
CRM platforms;
recruitment and HR systems;
customer service tools;
marketing platforms;
accounting or finance software;
cybersecurity products; and
industry-specific software.
Employees may also be using AI tools independently without the business having formally approved them.
A practical first step is therefore to create an AI register.
It does not need to be complicated. For each AI system, record basic information such as:
What tool are we using?
What are we using it for?
Who is using it?
What information goes into it?
What does it produce or influence?
Does it involve personal, confidential or commercially sensitive information?
Does a person review the output?
Who is the supplier?
You cannot properly manage AI risk if you do not know where AI is being used.
Step 2: Work out which uses actually matter
Not every use of AI needs the same level of governance.
This is where businesses can make AI governance unnecessarily difficult.
Instead of treating every AI system as high risk, look at the consequences of the particular use.
For example, using AI to suggest alternative wording for an internal presentation may present relatively limited risk.
Using AI to:
make or influence employment decisions;
assess customers;
process health or other sensitive information;
provide information customers rely upon;
analyse large amounts of personal information;
make financial recommendations;
generate important legal or compliance documents; or
make decisions without meaningful human review
requires much closer consideration.
Ask a simple question:
What could realistically go wrong if this AI system gets something wrong?
Then consider the potential impact.
Could someone suffer financial loss? Could confidential information be disclosed? Could personal information be mishandled? Could the business infringe someone else's intellectual property? Could an employee rely on inaccurate information? Could a customer be treated unfairly? Could the business make a decision it cannot properly explain?
The greater the potential impact, the stronger the controls should generally be.
Step 3: Look at the information going into AI systems
One of the most immediate legal issues is often not the AI output.
It is the information being put into the system in the first place.
Employees can easily copy information into an AI tool without thinking about what happens to that information afterwards.
That information might include:
customer information;
employee information;
commercially sensitive information;
source code;
internal financial information;
contracts;
intellectual property; or
information the business is contractually required to keep confidential.
Before approving an AI tool, understand what happens to the data entered into it.
For example:
Does the provider retain the data?
Can the provider use it to improve or train its AI models?
Where is the information stored?
Who can access it?
Can those uses be switched off?
What happens to the information when the contract ends?
This is also a privacy issue. The Office of the Australian Information Commissioner recommends that businesses conduct due diligence before adopting commercial AI products and consider matters including privacy, security, access to information and human oversight.
The OAIC also recommends, as a matter of best practice, that organisations do not enter personal information — particularly sensitive information — into publicly available generative AI tools because of the privacy risks involved.
Step 4: Review your AI suppliers — not just the technology
Many businesses will not build their own AI.
They will buy it.
That means AI governance is also a procurement and contracting issue.
Before introducing an AI-enabled product, understand both what the technology does and what the supplier's contract allows the supplier to do.
Depending on the system and the risks involved, this may include considering:
rights to use your data;
whether your data can be used to train AI models;
confidentiality;
privacy obligations;
cybersecurity requirements;
intellectual property ownership;
subcontractors and other third parties;
warranties about the AI system;
responsibility for inaccurate outputs;
audit or information rights;
liability if something goes wrong;
data return and deletion; and
what happens when the relationship ends.
This is an area businesses sometimes overlook.
An AI tool may appear technically suitable, but the contractual terms may create risks the business has not considered.
Step 5: Decide where humans need to remain involved
AI can assist with decisions.
That does not necessarily mean it should make them.
For higher-risk uses, businesses should clearly identify when human review is required and what that review actually involves.
Simply having a person click “approve” is not meaningful oversight if that person does not understand the system, cannot identify errors or does not have authority to challenge the outcome.
Consider:
Who reviews the AI output?
What are they expected to check?
Do they have enough information to identify when the output may be wrong?
Can they override the AI recommendation?
What happens if they disagree with it?
This becomes particularly important where AI affects employees, customers or other individuals.
Step 6: Give employees practical rules
Once you understand the risks, you can develop an AI policy.
But the policy should answer the questions employees actually have.
For example:
Can I use ChatGPT for work?
Which AI tools are approved?
Can I put client information into an AI tool?
Can I upload a contract?
Can I use AI to prepare a report for a customer?
Do I need to check AI-generated information?
Can I use AI-generated images or content?
Who do I ask before using a new AI tool?
A policy that simply says employees must use AI “responsibly, ethically and in accordance with applicable laws” does not give employees much practical guidance.
Good governance should make it easier for people to understand what they can and cannot do.
Step 7: Give someone responsibility for AI
AI governance should have an owner.
Who that is will depend on the business.
It may involve technology, legal, risk, privacy, cybersecurity, procurement or a combination of these functions.
For a mid-sized business, you may not need an AI committee with ten members and several layers of reporting.
But someone should be responsible for overseeing AI use and ensuring that higher-risk decisions are escalated appropriately.
There should also be a clear process for approving new AI systems and reviewing significant new uses of existing systems.
Without ownership, AI governance can quickly become everyone's responsibility — which in practice often means nobody is responsible.
Step 8: Don't treat AI governance as a one-off project
AI changes quickly.
So does the way employees use it.
A tool originally purchased for one purpose may later be used for something completely different. A supplier may introduce new AI functionality. Employees may start entering different types of information into the system. Contract terms may change.
Governance therefore needs some ongoing review.
That does not mean conducting a major legal review every month.
It can be as simple as periodically reviewing:
the AI register;
new AI tools and use cases;
higher-risk systems;
incidents or complaints;
changes to suppliers or their terms;
whether staff are following the AI policy; and
whether legal or regulatory requirements have changed.
You don't need to solve everything on day one
For many mid-sized businesses, AI governance feels overwhelming because the starting point is framed as:
“We need an AI governance framework.”
I would frame it differently.
Start with four questions:
1. Where are we using AI?
2. What information are we giving it?
3. What could go wrong with each use?
4. Who is responsible for managing that risk?
From there, you can determine what your business actually needs.
That might include an AI register, an AI policy, a process for assessing new AI tools, risk assessments for particular systems, changes to privacy documentation, employee training, contractual protections with AI suppliers, or more formal governance for higher-risk uses.
The objective should not be to create governance for the sake of governance.
It should be to allow the business to use AI in a way that is useful, commercially sensible and appropriately managed.
For many mid-sized businesses, that is the best place to start.
How Pixel Legal can help
Pixel Legal works with businesses on the legal and commercial issues that arise when adopting and using AI.
This can include understanding how AI is being used across the business, identifying higher-risk uses, developing practical AI governance frameworks and policies, reviewing AI suppliers and contracts, and advising on privacy, data and other legal risks associated with AI.
If your business is starting to think about AI governance — or AI is already being used and you are not quite sure what controls should be in place — get in touch with Pixel Legal to discuss where to start.
Disclaimer
Disclaimer: This article provides general information only and does not constitute legal advice. The legal and regulatory issues associated with AI will depend on how AI is used within your business and the circumstances of each use. You should obtain legal advice appropriate to your circumstances before acting on the information in this article.
