Record and replay Go API tests with Keploy

What you will build#

You will take a small Go URL shortener, record real HTTP calls against it with Keploy, and then replay those calls as automated tests. The shortener uses Gin for the API and MongoDB for storage.

Then you will stop MongoDB and run the tests again. They still pass, because Keploy recorded the database conversation too and plays it back.

I followed the "Running App Locally" variant of the Gin + MongoDB quickstart, which runs the app on the host and MongoDB in Docker.

What is Keploy#

Keploy is an open-source tool that writes API tests for you by watching real traffic. You start your app through Keploy, send requests to it, and Keploy saves each request and response as a test case. It also saves what your app said to its dependencies, such as a database, as mocks.

Later you run keploy test. Keploy sends the saved requests to your app and compares the new responses with the saved ones. Dependency calls are answered from the mocks, so the database does not need to be running.

Two terms come up a lot from here on:

• Test case: one recorded HTTP request and the response it got.
• Mock: one recorded exchange between your app and a dependency. Here that means MongoDB.

Prerequisites#

You need these installed before you start:

• Go. I installed it with brew install go.
• Docker Desktop, running. It only hosts MongoDB in this setup.
• git and curl.
• A free Keploy account. The first keploy record opens a browser window to sign in. More on that later.

Check the basics:

$ go version && docker --version && git --version

Get the sample app#

$ git clone https://github.com/keploy/samples-go.git && cd samples-go/gin-mongo

$ go mod download

r.GET("/:param", getURL)
r.POST("/url", putURL)
r.GET("/verify-email", verifyEmail)

Prepare the app#

The sample is set up to run inside Docker Compose, so its MongoDB host is a Compose service name. We run the app on the host instead, so it needs to point at localhost.

client, err := New("localhost:27017", dbName)

$ docker network create keploy-network

$ docker compose up mongo -d

$ go build -o test-app-url-shortener .

Install Keploy#

The installer script downloads the Keploy CLI for your system.

$ curl --silent -O -L https://keploy.io/install.sh && source install.sh

Then confirm it works:

$ keploy --version

I got Keploy 3.8.58. The binary is placed in ~/.keploy/bin. If your shell reports command not found, open a new terminal or add that folder to your PATH.

The sample repository ships with an old keploy/ folder of recorded tests. I moved it aside so the next steps start from a clean recording:

$ mv keploy keploy-shipped

Record test cases#

Recording means running your app under Keploy and using it normally. Keploy watches the traffic.

$ keploy record -c "./test-app-url-shortener"

INFO  Keploy is intercepting this application natively on macOS
INFO  Proxy started at port:16789
INFO  Keploy agent is ready to record test cases and mocks.
PID: 40740
[GIN-debug] POST   /url                      --> main.putURL (3 handlers)
INFO  Started ingress forwarding  {"orig_port": 8080, "new_port": 55405}

curl --request POST \
  --url http://localhost:8080/url \
  --header 'content-type: application/json' \
  --data '{"url": "https://google.com"}'
 
curl --request POST \
  --url http://localhost:8080/url \
  --header 'content-type: application/json' \
  --data '{"url": "https://keploy.io/docs"}'

{"ts":1790910561276549000,"url":"http://localhost:8080/Lhr4BWAi"}
{"ts":1790910561291329000,"url":"http://localhost:8080/PKSwnBrS"}

$ curl -s localhost:8080/url -H 'content-type: application/json' -d '{}'

$ curl -i localhost:8080/Lhr4BWAi

$ curl -s localhost:8080/doesnotexist

Replay the tests#

First stop MongoDB so it cannot help:

$ docker compose stop mongo

Then run the tests:

$ keploy test -c "./test-app-url-shortener" --delay 10

keploy test starts your app again, waits --delay seconds for it to be ready, and then sends each recorded request. Ten seconds is the value the quickstart uses, and the run time of about 10 seconds below reflects it.

This is the summary I got, with MongoDB stopped:

 TESTRUN SUMMARY. For test-set: "test-set-0"
	Total tests:        5
	Total test passed:  5
	Total test failed:  0
	Time Taken:         "10.05 s"

The app asked MongoDB questions. Keploy answered them from mocks.yaml.

Break a test on purpose#

A passing run does not prove much until you have seen one fail.

Testrun failed for testcase with id: "post-url-3"
 
│                    EXPECT BODY                    │                   ACTUAL BODY                     │
│  "error": "missing url" ,                         │  "error": "missing url param" ,                   │
 
	Total tests: 5
	Total test passed: 4
	Total test failed: 1

$ keploy test -c "./test-app-url-shortener" --delay 10

Read what Keploy generated#

Everything Keploy saves is plain YAML inside the keploy/ folder.

keploy/
├── .gitignore
├── reports/test-run-0/test-set-0-report.yaml
└── test-set-0/
    ├── config.yaml
    ├── mappings.yaml
    ├── mocks.yaml
    └── tests/
        ├── get-doesnotexist-1.yaml
        ├── get-lhr4bwai-1.yaml
        ├── post-url-1.yaml
        ├── post-url-2.yaml
        └── post-url-3.yaml

A test case#

This is the first request I recorded, trimmed to the interesting parts:

version: api.keploy.io/v1beta1
kind: Http
name: post-url-1
spec:
  req:
    method: POST
    url: http://localhost:8080/url
    body: '{"url": "https://google.com"}'
  resp:
    status_code: 200
    body: '{"ts":1790910561276549000,"url":"http://localhost:8080/Lhr4BWAi"}'
  assertions:
    noise:
      body.ts: []
      header.Date: []

The noise block tells Keploy to ignore fields that change on every call. The response has a timestamp in ts and a Date header, and neither would ever match twice. The quickstart tells you to add body.ts here yourself. In my run Keploy had already filled it in, probably during the automatic replay after recording.

The mocks#

mocks.yaml held nine entries, all of kind Mongo. The first few are the driver's connection handshake, marked type: config. The rest are the real queries: an update when a URL is shortened and a find when one is looked up.

mappings.yaml links tests to mocks. For example, post-url-1 uses mock-3. post-url-3, the empty request, has no mock at all, because the handler returns 400 before it ever talks to MongoDB.

How Keploy works#

Here is the model I ended up with, based on the logs and on Keploy's docs.

1. Keploy starts your app with a small interception layer attached. On Linux, Keploy's docs say this uses eBPF. On my Mac the log said intercepting this application natively on macOS, and the command line showed a DYLD_INSERT_LIBRARIES library being loaded into the app.
2. A local proxy sits between the outside world and your app. Requests to port 8080 pass through it, which is what the Started ingress forwarding line refers to.
3. When your app opens a connection to MongoDB, Keploy sees the wire traffic and stores each request and response pair.
4. On replay, Keploy plays the client role for HTTP. It sends the saved requests to your app. It plays the database role too, handing back the saved MongoDB responses.

Your code does not change, apart from the host name I edited earlier, which was needed for running outside Docker and has nothing to do with Keploy. There is no test helper to import and no mock to write.

Why this matters for Go developers#

Go makes unit testing pleasant, but the usual way to test a handler that calls a database is to define an interface, write a fake, and keep that fake in step with the real driver. This sample has none of that. The handler calls col.FindOne and col.UpdateOne on the MongoDB driver directly.

With Keploy:

• You get tests for existing code without refactoring it for testability.
• The tests come from real traffic, so they include the odd cases you actually hit, like the unknown hash that returns 404.
• They run fast and offline, because nothing real is called.
• Test cases are readable YAML files that you can review in a pull request.

There are trade-offs. Recorded tests describe what the app did when you recorded, not what it should do. If the first behaviour was wrong, Keploy will happily lock the wrong behaviour in. And when you change how the app queries MongoDB, the old mocks no longer match, so you re-record.

Common problems and fixes#

Two of these happened to me. The rest are mistakes the docs warn about, which I did not reproduce.

Problems I hit#

Mistakes to avoid#

These come from the quickstart and installation docs. I did not hit them, so treat them as guidance and not as war stories.

• Leaving the MongoDB host as mongoDb. That name only resolves inside the Docker Compose network. Running the app on the host needs localhost:27017.
• Using go run on macOS. The quickstart advises running a built binary there. Keploy's macOS notes also say wrapper commands such as npm start or a make recipe lose the interception, so give -c the real executable.
• Recording with an older Keploy and expecting the old file layout. The quickstart page still shows test-1.yml and a v1beta2 header. My version wrote post-url-1.yaml with v1beta1. The idea is the same, but file names differ.

Verification checklist#

Tick these off against your own run:

• keploy --version prints a version.
• docker ps shows mongoDB before recording.
• keploy record shows Keploy agent is ready to record test cases and mocks.
• Five test YAML files exist in keploy/test-set-0/tests/.
• mocks.yaml exists and contains kind: Mongo entries.
• MongoDB is stopped, and keploy test still reports all tests passed.
• Editing one expected value makes exactly one test fail.

Clean up#

When you are finished, tidy up from the gin-mongo folder.

$ docker compose down

$ docker network rm keploy-network

$ rm -rf keploy keploy.yml && mv keploy-shipped keploy && git checkout main.go

What you learned#

• Keploy turns live requests into test cases and the app's database calls into mocks.
• keploy record -c "<app command>" captures, and keploy test -c "<app command>" --delay <seconds> replays.
• Replay works without the database, which makes the tests quick and dependable.
• Fields that change on every call, like timestamps, belong in noise.
• Stopping a recording also runs a first replay, so you get feedback right away.

Next steps#

• Record more behaviour, for example the /verify-email route, and re-run the tests.
• Read the Keploy docs on recording filters and the configuration file.
• Re-run the suite after you change code. After my failing run, the CLI suggested keploy test -c './test-app-url-shortener' --mappings for re-running after a fix. Its notes on CI use the same command and expect a KEPLOY_API_KEY secret.
• Try the other Go quickstarts and compare how each database is mocked.