ChatGPT vs Gemini: One Synthetic Workflow Trial
Both products matched the expected answers in a single three-task synthetic workflow trial. See the input, scoring rules, settings and limits.
Test date: . One first attempt per product, with three tasks bundled into one prompt. All records were synthetic.
What we observed
Both returned a JSON object matching the expected structure and values. This small result does not establish a general winner, production reliability or value for money.
| Task | ChatGPT | Gemini |
|---|---|---|
| Appointment normalization | Matched | Matched |
| ROI calculation | Matched | Matched |
| Revenue stage separation | Matched | Matched |
Input and scoring
The expected answers were fixed before either submission. We compared parsed JSON for exact keys and values, preserved string IDs and list order, and rejected duplicate keys or non-JSON wrapping. Formatting whitespace was not scored. The same prompt text was entered in both interfaces; readback was checked after normalizing whitespace.
- Appointments: convert a fixed UTC+09:00 offset, retain the last valid duplicate, separate cancelled records, and flag missing times and invalid dates.
- ROI: use organization-wide costs and time savings once. Expected annual cost: JPY 394,000; benefit valued as time: JPY 240,000; net: JPY −154,000; ROI: −39.09%; payback: 31.25 months. Time value is not demonstrated cash savings.
- Revenue: keep clicks, occurred amounts, unpaid confirmations, cancellations and received cash separate.
Download the full prompt, expected answer and both captured responses (JSON)
Read the exact test prompt
Complete the three tasks below using only the supplied data and rules. Return exactly one JSON object with keys "appointments", "review", "cancelled_ids", "roi", and "revenue". No Markdown, explanation, extra keys, external research, or messages to anyone. All records are synthetic.
Task 1: Normalize appointments.
Each timestamp is local time at the explicitly fixed offset UTC+09:00. Validate calendar dates. Convert valid appointments to UTC as YYYY-MM-DDTHH:MM:SSZ. Never infer a missing time. A cancelled record is excluded from appointments and included in cancelled_ids. Put missing-time or invalid-date records in review, with reason exactly "missing_time" or "invalid_date". When several valid scheduled records share an id, keep the last valid scheduled record in input order. Sort appointments by UTC time ascending, and review/cancelled_ids by id ascending. Preserve id strings. Each appointment must have only id and utc; each review record only id and reason.
Input, in order:
{"id":"001","date":"2026-01-31","time":"23:30","status":"scheduled"}
{"id":"002","date":"2026-02-01","time":"00:15","status":"scheduled"}
{"id":"001","date":"2026-01-31","time":"23:45","status":"scheduled"}
{"id":"003","date":"2026-02-01","time":"09:00","status":"cancelled"}
{"id":"004","date":"2026-02-01","time":null,"status":"scheduled"}
{"id":"005","date":"2026-02-30","time":"10:00","status":"scheduled"}
{"id":"006","date":"2026-02-01","time":"08:00","status":"scheduled"}
Task 2: Calculate a 12-month ROI in JPY.
Initial setup is 200000 and initial training is 50000. Subscription is 12000 per month for the WHOLE organization. Time saved is 8 hours per month for the WHOLE organization, valued at 2500 JPY/hour. There are 4 employees; do not multiply whole-organization inputs by the employee count. Additional gross profit is 0.
annual_cost = initial setup + initial training + 12 * monthly subscription.
annual_benefit = 12 * (monthly hours saved * hourly value + monthly additional gross profit).
annual_net = annual_benefit - annual_cost.
roi_percent = annual_net / annual_cost * 100, rounded to 2 decimal places.
payback_months = (initial setup + initial training) / (monthly benefit - monthly subscription), rounded to 2 decimal places; null if the denominator is not positive.
Return roi with exactly annual_cost, annual_benefit, annual_net, roi_percent, payback_months, and benefit_type. Set benefit_type to "valued_time_not_cash_savings"; no cash saving has been established.
Task 3: Separate revenue stages.
The following events are distinct transactions, not status updates to one transaction. Sum amounts within each stage only. A click is not revenue. Do not add occurred, confirmed, cancelled and paid amounts into one revenue total.
{"id":"E1","stage":"click","amount":null}
{"id":"E2","stage":"occurred","amount":1000}
{"id":"E3","stage":"confirmed","amount":2000}
{"id":"E4","stage":"cancelled","amount":500}
{"id":"E5","stage":"paid","amount":1500}
Return revenue with exactly click_count, occurred_amount, unpaid_confirmed_amount, cancelled_amount, and received_cash_amount.
Settings and limitations
ChatGPT used an existing Pro account with “高” (High effort) visible. Its underlying model was not displayed. Gemini displayed “Pro 拡張” (Pro extended); the billing plan was not verified. Both used temporary chats. These were different existing account settings, so this is not a controlled comparison of identical plans or model tiers.
There were no retries, external research, real customer records or real transactions. We did not measure response latency, subscription costs, integrations, long-document handling, repeated-run accuracy or behavior on other inputs. Three tasks in one response are not three independent trials.
Using this result
Both handled this particular structured workflow correctly. Before choosing a product, repeat representative tasks with your own non-sensitive sample data and acceptance rules. This result provides no basis to pay more for either product.
These are ordinary official links; no verified referral commission is attached. Other catalog entries remain unverified unless their page states otherwise.