Skip to content

Preventing Duplicate GraphQL Mutations with the Angular Service Worker and AWS Amplify

A GraphQL mutation gets executed twice on the server even though the client sent it once. The symptom pattern: two records with byte-identical payloads but different server-generated IDs, written 2 to 4 seconds apart by two separate resolver invocations. Only in production. Never reproducible on demand. If your Angular app registers @angular/service-worker and talks to AppSync through AWS Amplify, this is not a bug in your code. It is an interaction between two framework defaults.

Why the usual suspects fail

Work through the standard explanations first. Each one can be ruled out with evidence the system already provides:

Hypothesis Killed by
The user triggered the action twice If the payload contains a client-side timestamp (new Date().toISOString()), compare it across the duplicate rows. Identical to the millisecond means one payload build. Two user interactions are at least ~40 ms apart.
Backend retry (Lambda, SQS, Step Functions) Different AppSync request IDs and different server-generated IDs. A backend retry reuses the same request.
Lambda throttling triggered an SDK retry Check CloudWatch: zero throttles and a 100% success rate in the relevant window rule this out.
Component code subscribed twice to the request observable client.graphql() returns an eagerly-created Promise. Wrapping it in from() shares that Promise. Re-subscription cannot produce a second HTTP request.

The identical client timestamp is the decisive constraint. The mutation payload was built exactly once. Yet the server received it twice, seconds apart. Something between the application code and the network re-sent a fully serialized request.

Two components sit in that gap: the Angular service worker and the Amplify client stack. Each is harmless alone. Together they duplicate writes.

Part 1: ngsw converts connection failures into 504 responses

An Angular service worker registered with ServiceWorkerModule.register('ngsw-worker.js') intercepts every fetch from the page, including cross-origin POSTs to AppSync. That is how the Service Worker spec works; interception is not opt-in per request.

For requests that match no assetGroups or dataGroups, ngsw passes the request through its safeFetch helper. This is the relevant code from ngsw-worker.js:

async safeFetch(req) {
  try {
    return await this.scope.fetch(req);
  } catch (err) {
    this.debugger.log(err, `Driver.fetch(${req.url})`);
    return this.adapter.newResponse(null, {
      status: 504,
      statusText: "Gateway Timeout"
    });
  }
}

If the underlying fetch throws, because the connection dropped, the network changed, or a proxy cut the socket, ngsw does not propagate the error. It fabricates an HTTP 504 response and hands it to the page. The application never learns that the network failed. It sees a server error that no server ever sent.

Part 2: Amplify silently retries 5xx responses, including mutations

The Amplify GraphQL client (tested with aws-amplify 6.1.4) routes POSTs through a retry middleware. The retry decision lives in @aws-amplify/core:

const getRetryDecider = (errorParser) => async (response, error) => {
  const parsedError = error ?? (await errorParser(response)) ?? undefined;
  const errorCode = parsedError?.code || parsedError?.name;
  const statusCode = response?.statusCode;
  return (isConnectionError(error) ||
    isThrottlingError(statusCode, errorCode) ||
    isClockSkewError(errorCode) ||
    isServerSideError(statusCode, errorCode));
};

const isServerSideError = (statusCode, errorCode) =>
  (!!statusCode && [500, 502, 503, 504].includes(statusCode)) ||
  (!!errorCode && TIMEOUT_ERROR_CODES.includes(errorCode));

A 504 is retried. Up to 3 attempts total, with a jittered backoff starting around 200 to 300 ms:

function jitteredBackoff(maxDelayMs = MAX_DELAY_MS) {
  const BASE_TIME_MS = 100;
  const JITTER_FACTOR = 100;
  return attempt => {
    const delay = 2 ** attempt * BASE_TIME_MS + JITTER_FACTOR * Math.random();
    return delay > maxDelayMs ? false : delay;
  };
}

The retry re-sends the already-serialized request body. Same payload, same client timestamp, new HTTP request. AppSync treats it as a fresh mutation.

There is an ironic detail. Amplify's own connection-error retry is broken in this version:

// The fetch handler throws this on a network failure:
throw new Error('Network error');   // .name === 'Error', .message === 'Network error'

// The retry decider checks the wrong property:
const isConnectionError = (error) => error?.name === 'Network error';  // never matches

Without the service worker, a dropped connection surfaces as an error to the application, the user sees a failure message, and no silent duplicate is possible. The service worker's fabricated 504 is what re-activates the retry path through the isServerSideError branch.

The full failure sequence

  1. The client sends a mutation whose server-side processing takes several seconds.
  2. The connection drops mid-request. The request already reached the server, which keeps processing and commits write #1.
  3. The service worker's fetch throws. safeFetch returns a synthetic 504 to the page.
  4. Amplify's retry middleware sees the 504 and re-sends the identical body after ~250 ms.
  5. The retry succeeds. The server commits write #2.
  6. The application receives one successful response and reports success. Nobody sees an error.

Long-running mutations are disproportionately exposed because they keep the connection open longest. Fast mutations rarely hit the drop window, which is why the problem clusters on specific operations and specific networks and why it looks random.

Reproduce it in two minutes

No network manipulation needed. Fabricate the 504 yourself and watch Amplify retry. Run this in the browser console of a production build, then trigger any mutation:

const origFetch = window.fetch;
let armed = true;
window.fetch = async function (...args) {
  const body = args[1]?.body ?? '';
  if (armed && typeof body === 'string' && body.includes('mutation')) {
    armed = false;
    origFetch.apply(this, args).catch(() => {});   // real request still goes out
    console.warn('>>> fabricated 504, real request already sent');
    return new Response(null, {status: 504, statusText: 'Gateway Timeout'});
  }
  return origFetch.apply(this, args);
};

Expected result: the Network tab shows a second, identical mutation ~250 to 700 ms after the first, and the database contains two records. That is exactly the signature the real connection drop produces in production.

To prove the service worker half, request a domain that cannot resolve. DNS failures are instant, so no timing is required:

fetch('https://does-not-exist-xyz-98765.invalid/test', {method: 'POST', body: 'x'})
  .then(r => console.log('response received:', r.status, r.statusText))
  .catch(e => console.log('error propagated:', e.message));

With the service worker controlling the page, this logs response received: 504 Gateway Timeout. Check Bypass for network in DevTools → Application → Service Workers and run it again: now it logs error propagated: Failed to fetch. The 504 comes from the service worker and nowhere else.

The fix: keep GraphQL out of the service worker

Angular provides an official escape hatch for exactly this situation: the ngsw-bypass header or query parameter. It is the very first check in the worker's fetch handler. With Amplify, one central place covers every GraphQL call:

import { Amplify } from 'aws-amplify';
import { fetchAuthSession } from 'aws-amplify/auth';
import config from './amplifyconfiguration.json';

Amplify.configure(config, {
  API: {
    GraphQL: {
      headers: async () => ({
        Authorization: (await fetchAuthSession()).tokens?.idToken?.toString() as string,
        'ngsw-bypass': 'true'
      })
    }
  }
});

Two things to verify after deploying:

  1. CORS preflight. The custom header must be allowed by the API's Access-Control-Allow-Headers. AppSync accepts it. If your gateway does not, append ?ngsw-bypass=true to the endpoint URL instead; ngsw treats the query parameter identically.
  2. Behavior on real failures. A dropped connection now surfaces as an honest error. The UI shows its failure state and the user retries deliberately, instead of the platform duplicating the write behind their back.

Nothing of value is lost. Without dataGroups, the service worker never cached or offline-handled these requests anyway. App-shell caching for scripts, styles, and assets continues to work unchanged.

The bypass removes the fabricated 504. Genuine 5xx responses from the API are still retried by Amplify, which is correct behavior for a server that has not processed the request. If your mutations are expensive enough that even legitimate retries hurt, add a client-generated idempotency key as a second line of defense.

Conclusion

A service worker that masks network failures as HTTP 504 and an HTTP client that silently retries 5xx responses are each defensible designs. Composed, they turn every mid-request connection drop into a duplicated mutation, invisible to the user, the application code, and the server logs. If you run @angular/service-worker together with AWS Amplify, or any client stack that retries on 5xx, send your mutations with ngsw-bypass. And when you hunt duplicates, check client-generated timestamps first: identical values prove the payload was built once and re-sent below your application code.

This analysis comes from debugging a production incident at Pantarey, where the fix now protects all GraphQL traffic across tenants.