MSC4140 in Synapse v1.141.0 Fails to Send Messages Containing Cyrillic Characters

[softpigeones]

Description

MSC4140 delayed events in Synapse fail when the message body contains non‑ASCII characters, resulting in M_UNKNOWN instead of a valid delay_id response.

Steps to reproduce

Run the following curl command:

curl -v "https://matrix.server.ru/_matrix/client/v3/rooms/!room:server.ru/send/m.room.message/DaewooK2?org.matrix.msc4140.delay=60000" \
  -X PUT \
  -d '{"body":"Тест отложенной отправки сообщений в Synapse!","msgtype":"m.text"}'

Expected Behavior

When sending a delayed event using MSC4140, the server should return a response like:

{"delay_id":"syd_akHesrlUxCRsJrXNojQW"}

Actual Behavior

Instead, the server responds with:

{"errcode":"M_UNKNOWN","error":"Internal server error"}

This occurs specifically when the message body contains Cyrillic characters.

Homeserver

Private HS

Synapse Version

v1.141.0

Installation Method

pip (from PyPI)

Database

PostgreSQL, single server, database not was ported from sqlite, no backups restored

Workers

Single process

Platform

Armbrian 25.8.0-trunk.411 (Debian 11), no containers.

Configuration

experimental_features:
  msc4140_enabled: true

max_event_delay_duration: 365d
caches:
  global_factor: 4.0
  event_cache_size: 10000
  response_cache_size: 5000
  user_cache_size: 5000
  room_cache_size: 3000
  federation_cache_size: 3000

Relevant log output

2025-11-30 23:24:16,469 - synapse.access.http.8008 - 476 - DEBUG - PUT-4 - **** - 8008 - Received request: PUT /_matrix/client/v3/rooms/!room:server/ru/send/m.room.message/SamsungK9?org.matrix.msc4140.delay=3595000
2025-11-30 23:24:16,525 - synapse.access.http.8008 - 521 - INFO - PUT-4 - *** - 8008 - {@me:server.ru} Processed request: 0.054sec/0.002sec (0.017sec, 0.004sec) (0.005sec/0.029sec/6) 55B 500 "PUT /_matrix/client/v3/rooms/!room:server.ru/send/m.room.message/SamsungK9?org.matrix.msc4140.delay=3595000 HTTP/1.1" "curl/8.16.0" [0 dbevts]

Anything else that would be useful to know?

• The issue seems related to handling of non‑ASCII (Cyrillic) characters in delayed events.
• Works as expected with plain ASCII text.

[devonh]

I tested this out locally and it fails repeatedly anytime there are Cyrillic characters in the event body.

[devonh]

It looks like the issue is here:

synapse/synapse/storage/databases/main/delayed_events.py

Line 150 in 978ae0b

"content": json_encoder.encode(content),

We store content as type bytea in the db. But our JSON string encoding isn't enough for this to work with non-ascii characters.
A fix might be to change the column type to text similar to how we store json in the event_json table.

As a temporary fix locally, I can get delayed events with Cyrillic working if I change that line to:

"content": bytearray(json_encoder.encode(content).encode('utf-8')),

But I'm not sure if that is the right way to go about things.
See matrix-org/synapse#6186 for more context.

@AndrewFerr Do you have thoughts on this issue since you were working on delayed events?

Here is the full stack trace:

Traceback (most recent call last):
  File "/home/cmdr/work/synapse/synapse/http/server.py", line 335, in _async_render_wrapper
    callback_return = await self._async_render(request)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cmdr/work/synapse/synapse/http/server.py", line 576, in _async_render
    callback_return = await raw_callback_return
                      ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cmdr/work/synapse/synapse/rest/client/room.py", line 478, in on_PUT
    return await self.txns.fetch_or_execute_request(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cmdr/work/synapse/.venv/lib/python3.12/site-packages/twisted/internet/defer.py", line 1187, in __iter__
    yield self
  File "/home/cmdr/work/synapse/.venv/lib/python3.12/site-packages/twisted/internet/defer.py", line 1853, in _inlineCallbacks
    result = context.run(
             ^^^^^^^^^^^^
  File "/home/cmdr/work/synapse/.venv/lib/python3.12/site-packages/twisted/python/failure.py", line 467, in throwExceptionIntoGenerator
    return g.throw(self.value.with_traceback(self.tb))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cmdr/work/synapse/synapse/rest/client/room.py", line 425, in _do
    delay_id = await self.delayed_events_handler.add(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cmdr/work/synapse/synapse/handlers/delayed_events.py", line 375, in add
    delay_id, next_send_ts = await self._store.add_delayed_event(
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cmdr/work/synapse/synapse/storage/databases/main/delayed_events.py", line 160, in add_delayed_event
    next_send_ts = await self.db_pool.runInteraction(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cmdr/work/synapse/synapse/storage/database.py", line 998, in runInteraction
    return await delay_cancellation(_runInteraction())
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cmdr/work/synapse/.venv/lib/python3.12/site-packages/twisted/internet/defer.py", line 1187, in __iter__
    yield self
  File "/home/cmdr/work/synapse/.venv/lib/python3.12/site-packages/twisted/internet/defer.py", line 1853, in _inlineCallbacks
    result = context.run(
             ^^^^^^^^^^^^
  File "/home/cmdr/work/synapse/.venv/lib/python3.12/site-packages/twisted/python/failure.py", line 467, in throwExceptionIntoGenerator
    return g.throw(self.value.with_traceback(self.tb))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cmdr/work/synapse/synapse/storage/database.py", line 964, in _runInteraction
    result: R = await self.runWithConnection(
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cmdr/work/synapse/synapse/storage/database.py", line 1100, in runWithConnection
    return await make_deferred_yieldable(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cmdr/work/synapse/.venv/lib/python3.12/site-packages/twisted/internet/defer.py", line 1187, in __iter__
    yield self
  File "/home/cmdr/work/synapse/.venv/lib/python3.12/site-packages/twisted/python/threadpool.py", line 269, in inContext
    result = inContext.theWork()  # type: ignore[attr-defined]
             ^^^^^^^^^^^^^^^^^^^
  File "/home/cmdr/work/synapse/.venv/lib/python3.12/site-packages/twisted/python/threadpool.py", line 285, in <lambda>
    inContext.theWork = lambda: context.call(  # type: ignore[attr-defined]
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cmdr/work/synapse/.venv/lib/python3.12/site-packages/twisted/python/context.py", line 117, in callWithContext
    return self.currentContext().callWithContext(ctx, func, *args, **kw)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cmdr/work/synapse/.venv/lib/python3.12/site-packages/twisted/python/context.py", line 82, in callWithContext
    return func(*args, **kw)
           ^^^^^^^^^^^^^^^^^
  File "/home/cmdr/work/synapse/.venv/lib/python3.12/site-packages/twisted/enterprise/adbapi.py", line 282, in _runWithConnection
    result = func(conn, *args, **kw)
             ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cmdr/work/synapse/synapse/storage/database.py", line 1093, in inner_func
    return func(db_conn, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cmdr/work/synapse/synapse/storage/database.py", line 820, in new_transaction
    r = func(cursor, *args, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cmdr/work/synapse/synapse/storage/databases/main/delayed_events.py", line 139, in add_delayed_event_txn
    self.db_pool.simple_insert_txn(
  File "/home/cmdr/work/synapse/synapse/storage/database.py", line 1151, in simple_insert_txn
    txn.execute(sql, vals)
  File "/home/cmdr/work/synapse/synapse/storage/database.py", line 458, in execute
    self._do_execute(self.txn.execute, sql, parameters)
  File "/home/cmdr/work/synapse/synapse/storage/database.py", line 520, in _do_execute
    return func(sql, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
psycopg2.errors.InvalidTextRepresentation: invalid input syntax for type bytea
LINE 1: ...EB:localhost:8008', 'm.room.message', NULL, NULL, '{"body":"...
                                                             ^

[AndrewFerr]

delayed_events.content should indeed use whatever the standard type is for event content. I may have seen bytea used for some other table column that stored JSON, but have no aversion to changing it now to fix this.

Maybe now is a good time to check if there are any other content-storing bytea columns that may hit this same issue.