Can’t ChatGPT simply create shift schedules too? We hear this question again and again. So, we put it to the test.
Our specialized shift planning AI against the all-rounder ChatGPT (Version 5.6). A real-world test with actual data from the care sector. The results even surprised us…
This was our basis: 6 employees, 10 days, the usual hard and soft constraints (these are must-rules and should-rules). So, a simple shift schedule, a task that Planerio’s automatic planning solves daily. Here are the results:
What Did ChatGPT Do Really Well?
ChatGPT explained complex relationships well.
All explanations were clear and comprehensible.
ChatGPT addressed fairness aspects itself.
Without us asking, ChatGPT suggested distributing night shifts fairly. An important aspect that many forget. However, this fairness was not implemented in the actually created schedule (see Problem 1).
ChatGPT recognized its own error.
After an error, we inquired. ChatGPT (almost) recognized and corrected the error itself. This ability for self-correction is remarkable.
ChatGPT itself warned that errors could occur during calculation.
“Errors can occur” and “There are better tools for this.”
However, the Following Problems Occurred:
Problem 1: Unfair Distribution
One carer was completely forgotten – they had 0 shifts in 10 days.
The other carers? Were massively overworked in some cases.
Problem 2: Compliance Violations
Weekly working hour limits were exceeded.
In real life, this means: legal consequences and a risk to employee health.
A plan that exceeds ArbZG limits is inadmissible – regardless of whether it was created by an AI or a human. Our article Shift planning with AI: What is allowed? shows which rules apply to AI use.
Problem 3: Hallucinated Rules
ChatGPT invented a “preference for day shifts” that we had never specified.
Problem 4: Mathematical Errors
The total number of shifts was counted incorrectly.
In the end, ChatGPT itself recommended using specialized tools like Planerio and did not complete the shift schedule despite instructions.
What Did We Learn from This?
LLMs are trained on pattern recognition, not exact calculations.
For duty and shift planning, this means:
- No guarantee of compliance
- No consistent fairness
- No reliability regarding planning targets
ChatGPT is brilliant at what it was developed for. But this example shows: Not every task is suitable for General Purpose AI.
Specialized systems like Planerio use deterministic algorithms. What does that mean? A look at the core characteristics of deterministic systems (Wikipedia):
What are the core characteristics of deterministic systems?
Determined: Future states are determined by the past.
Predictable: Identical inputs always lead to the same output.
No randomness: There are no true random elements or deviations in the process.
Our Findings Summarized:
General Purpose LLMs are an immense asset in many areas
General Purpose LLMs do not replace specialized systems
However, they can be combined: Specialized systems handle planning, LLMs can explain and analyze.
So, before you create your next shift schedule with ChatGPT, it’s better to speak with the expert.
Contact Us for A Demo.
Your Planning Deserves Better!
This article serves for general information and does not replace legal advice.