BigQuery Data Sync

Overview

Enterprise customers who run their own data warehouse usually want to pull data out of this platform for further analysis. This module lets them configure a data sync to Google BigQuery per LINE OA, covering credential setup, connection testing, inspection of the most recent sync, and on-demand sync runs.

Every route in this module sits on the public group behind JwtLoginAuth and carries no policy annotation at all — behavior inherited directly from the original NestJS code, which never applied @CheckPolicies here.

Business Flow

1. Read the current configuration — GET /api/bigquery-sync-config returns the settings bound to the current LINE OA.
2. Test before saving — POST /api/bigquery-sync-config/test-connection with TestConnectionDto attempts a BigQuery connection using the supplied credentials and returns a success flag plus a message, so the user knows the configuration is valid before committing it.
3. Save the configuration — POST /api/bigquery-sync-config with CreateBigquerySyncConfigDto, specifying the project id, dataset, credentials, the tables to sync, and the schedule.
4. Edit the configuration — PUT /api/bigquery-sync-config/:id
5. Run a sync now — POST /api/bigquery-sync-config/:id/sync-now returns a success flag and a message.
6. Check the latest run — GET /api/bigquery-sync-config/:id/sync-status returns lastSyncAt, a nullable timestamp, and lastSyncResult, the raw JSON recorded from the most recent run.
7. Surface it in the UI — cms-web polls the status endpoint to display sync progress and any errors.

Key Files & Functions

The code lives in internal/modules/bigquerysync/, made up of controller.go, service.go, and dto.go. The BigQuery client itself is in internal/bigqueryx/bigqueryx.go and is reached through Deps.BigQuery.

Method	Route	Handler	Guard
GET	/api/bigquery-sync-config	ct.getConfig	JwtLoginAuth (public group)
POST	/api/bigquery-sync-config	ct.create	JwtLoginAuth
POST	/api/bigquery-sync-config/test-connection	ct.testConnection	JwtLoginAuth
PUT	/api/bigquery-sync-config/:id	ct.update	JwtLoginAuth
POST	/api/bigquery-sync-config/:id/sync-now	ct.syncNow	JwtLoginAuth
GET	/api/bigquery-sync-config/:id/sync-status	ct.getSyncStatus	JwtLoginAuth

The relevant response structures are TestConnectionResponse (fields success, message) and SyncStatusResponse (fields lastSyncAt, lastSyncResult), with field ordering kept identical to the original TypeScript object literals.

Connections to Other Services

• Access control — JwtLoginAuth only, with no policy check and no ModuleGate. This is worth keeping in mind when designing permissions: any user who can log in can call these endpoints.
• Tables — bigquery_sync_config and line_oa
• External services — Google BigQuery through the internal/bigqueryx package, which does not connect at boot but creates an empty client up front.
• CLS — Uses lineOaId and organizationId to scope data.
• Related modules — LINE User Management, the source data being synced, and All Friend Listing.