Modify

Opened 4 months ago

Last modified 3 months ago

#11953 new defect

OverflowError: regular expression code size limit exceed

Reported by: rjollos Owned by: rjollos
Priority: normal Component: AutoWikifyPlugin
Severity: normal Keywords:
Cc: ChrisNelson Trac Release:

Description (last modified by rjollos)

It was reported in trac:#10190 that the regular expression can become too large on a site with many wiki page.

A similar issue was also reported this week for the AcronymsPlugin in a private email exchange. The two plugins are very similar, so if we resolve the issue for one, we may have insight into solving the issue for the other.

Attachments (0)

Change History (9)

comment:1 Changed 4 months ago by ChrisNelson

  • Cc ChrisNelson added; anonymous removed

comment:2 follow-up: Changed 4 months ago by ChrisNelson

What I find odd is that this broke on our upgrade from Trac 0.11.6 to 1.0.1. I believe our version of Python and this plugin remained the same.

comment:3 Changed 4 months ago by rjollos

  • Description modified (diff)

comment:4 in reply to: ↑ 2 Changed 4 months ago by ChrisNelson

Replying to ChrisNelson:

What I find odd is that this broke on our upgrade from Trac 0.11.6 to 1.0.1. I believe our version of Python and this plugin remained the same.

BTW, this works in my sandbox but fails in production. Both are running the current 0.2dev Acronyms plugin, Trac 1.0.1 and the same version of Python. I'm trying to identify differences.

comment:5 Changed 4 months ago by jun66j5

I tried to reproduce this issue on Trac 1.0.1 with AutoWikifyPlugin and AcronymsPlugin. However, I cannot at the moment. I've created 83,000 wiki pages and installed the same of AcronymDefinitions in trac-hacks.org.... Could someone provide how to reproduce?

comment:6 Changed 4 months ago by ChrisNelson

I now suspect it's a plugin interaction. Our production has more plugins installed than my sandbox. I'm going to work on closing that gap shortly.

comment:7 follow-up: Changed 4 months ago by ChrisNelson

I'm more sure of a plugin interaction, or at least pile up. When I do:

  • sre_compile.py

    old new  
    504504    code = _code(p, flags)
    505505
    506506    # print code
     507    file = open("/tmp/re.txt", "a");
     508    file.write("re groups = %d\n" % p.pattern.groups)
     509    file.close
    507510
    508511    # XXX: <fl> get rid of this limitation!
    509512    if p.pattern.groups > 100:

I have 98 groups with Acronyms Plugin enabled and 95 without so Acronyms is only adding three. In my production environment where this fails, I have 98 without, 101 with, thus crossing the threshold.

Last edited 4 months ago by ChrisNelson (previous) (diff)

comment:8 in reply to: ↑ 7 ; follow-up: Changed 4 months ago by jun66j5

I have 98 groups with Acronyms Plugin enabled and 95 without so Acronyms is only adding three. In my production environment where this fails, I have 98 without, 101 with, thus crossing the threshold.

In that case, I guess that AssertionError: sorry, but this version only supports 100 named groups is raised, not OverflowError.

>>> import re
>>> re.compile('|'.join('(?P<i%d>i)' % _ for _ in xrange(99)))
<_sre.SRE_Pattern object at 0x00B4A148>
>>> re.compile('|'.join('(?P<i%d>i)' % _ for _ in xrange(100)))
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "C:\usr\apps\python24\lib\sre.py", line 180, in compile
    return _compile(pattern, flags)
  File "C:\usr\apps\python24\lib\sre.py", line 225, in _compile
    p = sre_compile.compile(pattern, flags)
  File "C:\usr\apps\python24\lib\sre_compile.py", line 506, in compile
    raise AssertionError(
AssertionError: sorry, but this version only supports 100 named groups

comment:9 in reply to: ↑ 8 Changed 3 months ago by ChrisNelson

Replying to jun66j5:

I have 98 groups with Acronyms Plugin enabled and 95 without so Acronyms is only adding three. In my production environment where this fails, I have 98 without, 101 with, thus crossing the threshold.

In that case, I guess that AssertionError: sorry, but this version only supports 100 named groups is raised, not OverflowError. ...

Yes, you are right. My issue appears to be different from the subject of this ticket.

Add Comment

Modify Ticket

Action
as new The owner will remain rjollos.
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.