Core HTTP, Middleware Pipeline & Database Access

Overview

This page describes the foundation every webhook-go endpoint stands on: how the gin engine is assembled, the middleware order, the error format, rate limiting, and the unusual Postgres connection setup that fixes a critical porting bug.

The skeleton comes from the company's go-service-template. The repository's README.md and CLAUDE.md still describe the template and have not been rewritten — the document that matches the real service is docs/PORTING_PLAN.md. The template has been trimmed down to API mode only: there is a cmd/api but no cmd/worker, no consumers, and no sample module.

Business Flow

Middleware order, per server.New

1. MetricsMiddleware() — the outermost layer, recording status codes and latency for every request (RED metrics).
2. gin.Logger() — enabled only outside production.
3. middleware.CORSAllowAll() — unconditionally allows every origin, method, and header, for parity with the original. An allowlist-based CORS(...) exists in the code but is never called.
4. middleware.Helmet() — the standard security header set, minus Cross-Origin-Resource-Policy, because the original set crossOriginResourcePolicy: false.
5. httpx.ExceptionMiddleware(sentry, errLog) — converts errors and panics into the standard envelope.

Two more layers are added at the /api group level:

6. clsx.Middleware() — builds the request context: transactionId (from X-Request-Id, or a random 12-character hex string), sessionId (the third segment of the Authorization header), and lang (from x-lang or the CLS_LANG setting). It also echoes X-Request-Id back in the response headers.
7. middleware.AppRateLimit(rdb, ttl, limit, block) — per-IP rate limiting.

Rate limiting

• The default is 200 requests per 1000 ms per IP, configured via RATE_LIMIT_APP_TTL=1000, RATE_LIMIT_APP_LIMIT=200, and RATE_LIMIT_BLOCK_DURATION=0.
• State lives in Redis under throttle:app:hits:{ip} and throttle:app:block:{ip}. The NestJS original kept counters in per-pod memory; moving them to Redis makes the limit effective across replicas.
• The client IP is resolved in order from the first x-forwarded-for entry, then x-real-ip, then x-client-ip, and finally the socket address.
• A Redis failure fails open — traffic is never blocked because the storage layer is unhealthy.
• Exceeding the limit returns 429 in envelope form. If blockDuration is configured, the offending IP stays blocked for that period.
• A RouteRateLimit helper exists for per-route overrides but is not used in this service.

The error envelope

Every error is emitted in the standard NestJS shape:

{
  "statusCode": 400,
  "message": "a string, or an array of strings",
  "error": "Bad Request"
}

• An unknown route returns 404 with {"statusCode":404,"message":"Cannot POST /xxx","error":"Not Found"}.
• Anything that is not an *httpx.Exception — including panics — becomes 500 "System Failure".
• httpx.NewException(status, code, message) discards the code argument, because the NestJS envelope has no code field. This matters when comparing behaviour against cms-api, whose envelope differs.
• Validation errors go through httpx.BindAndValidate together with a Rules() method — a port of class-validator covering IsArray, IsNotEmpty, IsBoolean, custom checks, and transforms.

Connecting to Postgres — "the binding bug fix"

This is the single most important technical decision in the entire port.

• pgx defaults to the binary protocol, which is strict about type OIDs. Passing a Go string into an enum column, an int into a text column, or a JSON string into jsonb all fail to bind.
• Node's pg and TypeORM, by contrast, send everything as text and let Postgres coerce the types.
• The fix is to open the pool with connCfg.DefaultQueryExecMode = pgx.QueryExecModeSimpleProtocol in internal/platform/db/db.go. This restores Node-equivalent semantics and is additionally safe under PgBouncer transaction pooling.

Rules every repository must follow:

• All ids and foreign keys are int64 — never int32.
• Nullable columns are pointers.
• Timestamps are timestamptz.
• JSONB columns must implement sql.Scanner and driver.Valuer.
• Enums bind as strings.
• Use $1, $2 placeholders and name columns explicitly — SELECT * is forbidden.
• sql.ErrNoRows maps to (nil, nil) and nothing else; every other error must propagate upward and must never be swallowed.

Bootstrap

cmd/api/main.go calls app.New(ctx, cancel, service, "api"), which loads and validates the configuration, builds the logger, connects every dependency (the sqlx pool, named Redis clients, the AMQP publisher), opens the admin port on :9100, and starts the supervisor. Control then returns to build the gin engine and serve on APP_PORT, which defaults to 6570.

Key Files & Functions

File	Highlights
cmd/api/main.go	main() — bootstrap, assembling server.Deps, declaring topology, serving, graceful shutdown
internal/app/app.go	app.New, struct App, App.Shutdown, App.ReloadLevel
internal/server/server.go	server.New(deps, registers...), struct Deps, (*Deps).APIKeyAuth, type RegisterFunc
internal/server/metrics_middleware.go	MetricsMiddleware()
internal/middleware/cors.go	CORSAllowAll() (the one in use), CORS(...), OriginGuard(...)
internal/middleware/helmet.go	Helmet()
internal/middleware/ratelimit.go	AppRateLimit(...), RouteRateLimit(...), tooManyRequests()
internal/middleware/apikey.go	api-key auth; see API Key Authentication
internal/clsx/clsx.go	Middleware(), Store, TransactionID/SessionID/Lang, LogFields
internal/httpx/exception.go	Exception, New, BadRequest/Unauthorized/NotFound/..., SystemFailure, NewException
internal/httpx/abort.go, validate.go, checks.go	Abort, BindAndValidate, and the Rule / Check set
internal/platform/db/db.go	sqlx over pgx stdlib with the simple query protocol
internal/platform/redisx/redisx.go	Manager for multiple Redis instances
internal/config/config.go, api.go, api_load.go	Typed configuration with fail-fast Validate()
internal/logx/logx.go	JSON slog, LOG_LEVEL, SIGHUP reload
cmd/api/smoke_test.go, wiring_test.go	Boot the real service and hit routes to verify the contract

Routes the platform mounts itself

Method	Route	Source
GET	/api	health.Register — returns the app name from APP_NAME
GET	/api/health	health.Register — Terminus-style response
GET	/livez, /readyz, /healthz	server.New on the business port; see Health Checks & Dependency Monitoring

Connections to Other Services

• Postgres — the api_client, api_key, audience, and line_oa tables. The live schema is captured in docs/live_schema.tsv (5 tables, 102 columns); line_user is declared but never queried.
• Redis — used both as a cache (see Webhook Config Cache) and as the rate limiter backend, in separate key spaces.
• RabbitMQ — see RabbitMQ Publisher & Topology.
• The service boots even with no backends at all — typed dependencies are simply nil, every handler guards for that, and all routes are still mounted, which makes wiring testable without any infrastructure.
• In-repo reference documents: docs/PORTING_PLAN.md (the complete porting plan and its status) and docs/live_schema.tsv.