Customer Database from CSV

Overview

Customer Database lets customers upload their own customer records as CSV files and store them in the system as a separate reference table. The data serves two main purposes: it backs profile mapping when users fill in forms, and it lets customers reconcile their existing records against their LINE friends.

Two technical details are worth knowing:

• CSV parsing mirrors JavaScript's papaparse precisely, including its quirks — a single-column CSV produces an UndetectableDelimiter error, and whenever an error occurs the resulting rows collapse to an empty array.
• Soft delete here is status-based: records are marked with status = 'delete' rather than using the deleted_date column that other modules rely on.

Business Flow

1. Preview first — POST /api/customer-database/preview accepts a multipart/form-data upload, parses the header row and a sample of records, and returns them so the user can verify the file is correct. A single-column CSV fails with UndetectableDelimiter, matching papaparse's original behaviour.
2. Create the database — POST /api/customer-database saves metadata to the customer_database table, then bulk-inserts the data rows into customer_database_row in batches of 1,000 within a single transaction.
3. List — GET /api/customer-database returns a paginated list scoped by lineOaId, excluding rows where status = 'delete'. It also reports which forms currently use each database, resolved through a raw query against the JSON column profile_mapping ->> 'databaseId' on the form_builder table.
4. View details — GET /api/customer-database/:id returns both the metadata and the stored rows.
5. Update — PUT /api/customer-database/:id accepts form-data and replaces every row within a single transaction.
6. Delete — DELETE /api/customer-database/:id sets status to delete.
7. When an end user fills in a form bound to this database, client-api looks up the matching row to auto-fill fields or verify eligibility.

Key Files & Functions

The code lives in internal/modules/customerdatabase/.

File	Role
controller.go	Route registration
service.go	Core logic — _getOwnedDatabase, _getMappedFormsMap, and transactional create/update
papaparse.go, papaparse_config.go	The papaparse-compatible CSV parser
orderedmap.go, orderedrow.go	Preserve column order so JSON output matches the legacy system
dto.go	DTO definitions

Method	Route	Handler	Policy
GET	/api/customer-database	ct.findAll	readAll customer-database
GET	/api/customer-database/:id	ct.findOne	read customer-database
POST	/api/customer-database	ct.create	create customer-database
POST	/api/customer-database/preview	ct.previewCsv	create customer-database
PUT	/api/customer-database/:id	ct.update	update customer-database
DELETE	/api/customer-database/:id	ct.remove	delete customer-database

Every route is wrapped in modulegate.ModuleGate(d, "customer-database").

Connections to Other Services

• Permissions — requires ModuleGate("customer-database"), meaning the organisation must have the module enabled, and carries PolicyModuleCustomerDatabase as metadata.
• Tables — customer_database, customer_database_row, form_builder (for mapping) and line_oa.
• Storage — the original CSV files are kept in object storage.
• CLS — lineOaId from the request context scopes every query.
• Related modules — Form Builder consumes this data, and Import Mapping offers an alternative import path that writes directly into the line_user table.