psycopg2.errors.ProgramLimitExceeded: string is too long for tsvector (1054410 bytes, max 1048575 bytes)

[valentijnscholten]

Occasionally for larger text fields we run into the limits of a postgres TSV vector.

Is there any way Django Watson could truncate fields or increase the max size for the vector in the schema generated by Django (Watson)?

[23/Jun/2023 12:40:22] INFO [django.request:241] OK: /product/20/import_scan_results
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 89, in _execute
    return self.cursor.execute(sql, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
psycopg2.errors.ProgramLimitExceeded: string is too long for tsvector (1054410 bytes, max 1048575 bytes)
CONTEXT:  PL/pgSQL function watson_searchentry_trigger_handler() line 3 at assignment


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/django/core/handlers/exception.py", line 56, in inner
    response = get_response(request)
               ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/utils/deprecation.py", line 138, in __call__
    response = self.process_response(request, response)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/watson/middleware.py", line 37, in process_response
    self._close_search_context(request)
  File "/usr/local/lib/python3.11/site-packages/watson/middleware.py", line 33, in _close_search_context
    search_context_manager.end()
  File "/usr/local/lib/python3.11/site-packages/watson/search.py", line 260, in end
    _bulk_save_search_entries(
  File "/usr/local/lib/python3.11/site-packages/watson/search.py", line 210, in _bulk_save_search_entries
    SearchEntry.objects.bulk_create(search_entry_batch)
  File "/usr/local/lib/python3.11/site-packages/django/db/models/manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/models/query.py", line 816, in bulk_create
    returned_columns = self._batched_insert(
                       ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/models/query.py", line 1817, in _batched_insert
    self._insert(
  File "/usr/local/lib/python3.11/site-packages/django/db/models/query.py", line 1791, in _insert
    return query.get_compiler(using=using).execute_sql(returning_fields)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 1660, in execute_sql
    cursor.execute(sql, params)
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 67, in execute
    return self._execute_with_wrappers(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 80, in _execute_with_wrappers
    return executor(sql, params, many, context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 84, in _execute
    with self.db.wrap_database_errors:
  File "/usr/local/lib/python3.11/site-packages/django/db/utils.py", line 91, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 89, in _execute
    return self.cursor.execute(sql, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
django.db.utils.OperationalError: string is too long for tsvector (1054410 bytes, max 1048575 bytes)
CONTEXT:  PL/pgSQL function watson_searchentry_trigger_handler() line 3 at assignment