Skip to content

feat(search): migrate unified search from Elasticsearch to PostgreSQL FTS pipeline#209

Open
hyunbin1 wants to merge 2 commits intomainfrom
codex/analyze-existing-elasticsearch-logic
Open

feat(search): migrate unified search from Elasticsearch to PostgreSQL FTS pipeline#209
hyunbin1 wants to merge 2 commits intomainfrom
codex/analyze-existing-elasticsearch-logic

Conversation

@hyunbin1
Copy link
Copy Markdown
Member

Motivation

  • Replace the Elasticsearch-based unified search pipeline with a self-contained PostgreSQL Full-Text Search implementation while keeping existing API signatures to minimize frontend changes.
  • Preserve existing domain preprocessing and ranking intent but map ES boost/signals to tsvector weights and SQL ranking formulas suitable for Korean text and operational constraints.
  • Keep Redis as a caching/fallback layer and provide a migration path (DDL, triggers, and rebuild) to remove ES from infra later.

Description

  • Added a new JDBC-backed repository PostgresUnifiedSearchRepository that implements unified search, autocomplete and top-keywords using PostgreSQL FTS and ranking SQL; file: src/main/java/nova/mjs/domain/thingo/ElasticSearch/Repository/PostgresUnifiedSearchRepository.java.
  • Switched service wiring so UnifiedSearchService now delegates to the Postgres repository while preserving the existing search(keyword, type, category, order, pageable) API; file: src/main/java/nova/mjs/domain/thingo/ElasticSearch/Service/UnifiedSearchService.java.
  • Reworked SearchIndexSyncService to build a full thingo_search_document table in Postgres from domain repositories and refresh FTS vectors instead of creating ES indices; file: src/main/java/nova/mjs/domain/thingo/ElasticSearch/Service/SearchIndexSyncService.java.
  • Converted SuggestService to call the Postgres autocomplete SQL ranking (personal/global/time-decay) and updated RealtimeKeywordService to persist search logs to Postgres while retaining Redis ZSET behavior for hot cache/fallback; files: SuggestService.java, RealtimeKeywordService.java.
  • Added Postgres schema, indexes, trigger and materialized view SQL for FTS, autocomplete and realtime keywords at src/main/resources/db/search/postgres_search_schema.sql and documented the migration, architecture, ranking and verification plan in docs/postgresql-search-migration-plan.md.

Testing

  • Attempted a build check with ./gradlew compileJava to validate compilation, but the build could not run in this environment due to a Gradle/JDK incompatibility reporting Unsupported class file major version 69 (build did not complete).
  • No automated unit/integration tests were executed in this environment after the changes; manual validation and CI build are required to confirm compilation and runtime behavior against the target JDK and Postgres 15+ environment.

Codex Task

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6bf2d022e9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".


//time stamp 리스트에 추가
redisTemplate.opsForList().rightPush(LIST_KEY_PREFIX + keyword, String.valueOf(now));

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Guard search-log writes so detail search still succeeds

recordSearch now performs a PostgreSQL insert on the request path without handling failures. If search_query_log is not yet migrated or the DB write fails transiently, this exception bubbles out of /api/v1/search/detail after search results are computed, turning successful searches into 500s. This logging path should be best-effort (catch/log and continue) so analytics outages do not break user-facing search.

Useful? React with 👍 / 👎.

}

public void rebuildSearchDocuments(List<SearchWriteModel> rows) {
jdbcTemplate.update("TRUNCATE TABLE thingo_search_document", new MapSqlParameterSource());
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Make table rebuild atomic to avoid empty search index

rebuildSearchDocuments truncates thingo_search_document before repopulating it, but this destructive step is not wrapped in a transaction with the subsequent batch insert. Any mid-rebuild failure (constraint error, timeout, connection drop) can leave the table empty or partially loaded, causing a broad search outage until a manual rerun. The truncate+insert path should run atomically (or use a staged swap).

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant