ProgramLimitExceeded: index row size 3696 exceeds maximum 2712 for index "session_attribute_pk"

[Ryan J Ollos]

Found in the logs:

2019-08-13 09:32:07,762 Trac[main] ERROR: [1.10.189.156] Internal Server Error: <RequestWithSession "POST '/register'">, referrer 'https://trac-hacks.org/register'¬
Traceback (most recent call last):¬
  File "/srv/trac-hacks.org/pve/local/lib/python2.7/site-packages/trac/web/main.py", line 648, in _dispatch_request¬
    dispatcher.dispatch(req)¬
  File "/srv/trac-hacks.org/pve/local/lib/python2.7/site-packages/trac/web/main.py", line 249, in dispatch¬
    resp = chosen_handler.process_request(req)¬
  File "/srv/trac-hacks.org/pve/local/lib/python2.7/site-packages/acct_mgr/register.py", line 428, in process_request¬
    acctmgr.validate_account(req, True)¬
  File "/srv/trac-hacks.org/pve/local/lib/python2.7/site-packages/acct_mgr/api.py", line 377, in validate_account¬
    inspector.validate_registration(req)¬
  File "/srv/trac-hacks.org/pve/local/lib/python2.7/site-packages/tracspamfilter/accountadapter.py", line 76, in validate_registration¬
    FilterSystem(self.env).test(req, author, changes)¬
  File "/srv/trac-hacks.org/pve/local/lib/python2.7/site-packages/tracspamfilter/filtersystem.py", line 329, in test¬
    self.reject_handler.reject_content(req, msg)¬
  File "/srv/trac-hacks.org/pve/local/lib/python2.7/site-packages/tracspamfilter/captcha/api.py", line 128, in reject_content¬
    req.redirect(req.href.captcha())¬
  File "/srv/trac-hacks.org/pve/local/lib/python2.7/site-packages/trac/web/api.py", line 671, in redirect¬
    self.send_response(status)¬
  File "/srv/trac-hacks.org/pve/local/lib/python2.7/site-packages/trac/web/main.py", line 111, in send_response¬
    self.session.save()¬
  File "/srv/trac-hacks.org/pve/local/lib/python2.7/site-packages/trac/web/session.py", line 200, in save¬
    for k, v in items])¬
  File "/srv/trac-hacks.org/pve/local/lib/python2.7/site-packages/trac/db/util.py", line 146, in executemany¬
    cursor.executemany(query, params)¬
  File "/srv/trac-hacks.org/pve/local/lib/python2.7/site-packages/trac/db/util.py", line 92, in executemany¬
    return self.cursor.executemany(sql_escape_percent(sql), args)¬
ProgramLimitExceeded: index row size 3696 exceeds maximum 2712 for index "session_attribute_pk"¬
HINT:  Values larger than 1/3 of a buffer page cannot be indexed.¬
Consider a function index of an MD5 hash of the value, or use full text indexing.¬
ProgramLimitExceeded: index row size 3696 exceeds maximum 2712 for index "session_attribute_pk"¬
HINT:  Values larger than 1/3 of a buffer page cannot be indexed.¬
Consider a function index of an MD5 hash of the value, or use full text indexing.¬

[Jun Omae]

I think that is spam-filter plugin's issue.

The plugin saves submitted form data in session_attribute table at ​trac:source:plugins/1.2/spam-filter/tracspamfilter/captcha/api.py@15250:126-127#L118. When a spam bot submits form data with long item name, 'captcha_arg_%s' % key is used for name column. As the result, index of the session_attribute record will be exceeded the maximum.

We could use req.session['captcha_args'] = to_json(req.args) to avoid using item name and the exceeding maximum.

             req.session['captcha_redirect'] = req.path_info
-            for key, value in req.args.iteritems():
-                req.session['captcha_arg_%s' % key] = value
+            req.session['captcha_args'] = to_json(req.args)
             req.redirect(req.href.captcha())

Another thing, I think we should probably store req.arg_list because req.args can be modified by pre_process_request of components.

[Jun Omae]

I consider we should prevent to use tainted data, e.g. req.args, for key of req.session.

trac=> INSERT INTO session_attribute
trac-> VALUES ('868d27e2d2fa95460f6cd49d', 0,
trac->         repeat('5OwturOtwedNewvyurd5', 20000),
trac->         'value');
ERROR:  index row size 4656 exceeds maximum 2712 for index "session_attribute_pk"
HINT:  Values larger than 1/3 of a buffer page cannot be indexed.
Consider a function index of an MD5 hash of the value, or use full text indexing.

[Ryan J Ollos]

Replying to Jun Omae:

I consider we should prevent to use tainted data, e.g. req.args, for key of req.session.

It looks like the args are stored in the session and then deleted after the redirect: ​source:plugins/1.2/spam-filter/tracspamfilter/captcha/api.py@15250:188-199#L163. So we can just send the arguments on redirect and avoid storing in the session?