Opened 12 years ago

Last modified 4 years ago

#388 new enhancement

Ability to to perform re-index with a quota on time / memory consumption

Reported by: Patrick Martin Owned by: anonymous
Priority: normal Component: RepoSearchPlugin
Severity: normal Keywords:
Cc: Trac Release: 0.9


I don't think this is an easy one, but the time and resources required to re-index a large number of changes are quite significant.

Is it possible for a re-index to index a configurable number of files on each invocation? This would leave the repository search out of date, but release the server to serve the requests of other users. Susbsequent re-indexes could fill in the gap.

I'm not sure how valuable this would be if #365 were implemented, as one huge cost to re-indexing (tagging and branching) could be radically reduced. Nonetheless, repository search is transparently such a great feature is would be a shame if it became a victim of its own success when people with large-ish source trees (>1K files) have a go at using it.

Attachments (0)

Change History (9)

comment:1 Changed 12 years ago by Alec Thomas

Status: newassigned

The massive consumption of memory during indexing is a definite issue. In the near future I will be porting RepoSearchPlugin to use the pyndexter indexer abstraction layer. After this is complete I will probably rewrite the default indexer (essentially what is currently used in this plugin) as a C extension. These two changes should alleviate most performance related problems, particularly if #365 is implemented.

comment:2 Changed 12 years ago by Patrick Martin

This sounds like great stuff.

Can't wait.

comment:3 Changed 11 years ago by Alec Thomas

(In [1881]) Not for the faint of heart, but if anybody wants to test the pyndexter branch it would be most useful to get some feedback.

You will need the refactoring branch of pyndexter, available from here.

More info in the README.

References #362, #371, #385 and #388.

comment:4 Changed 8 years ago by Ryan J Ollos

Reassigning ticket after changing maintainer of plugin.

comment:5 Changed 8 years ago by Ryan J Ollos

Owner: changed from Alec Thomas to Ryan J Ollos
Status: assignednew

comment:6 Changed 8 years ago by Ryan J Ollos

Summary: ability to to perform re-index with a quota of time / memory consumptionAbility to to perform re-index with a quota on time / memory consumption

Related to #385 (which is probably what is meant when referring to #365).

comment:7 Changed 7 years ago by Ryan J Ollos


I took over maintainership of this plugin from athomas some time ago. There is a significant amount of work to do on this plugin, and I don't foresee having the time to do it all.

helend has written the TracSuposePlugin, which seems like a much better solution. Rather than writing the repository search functionality from scratch, a Trac interface to an existing repository search tool has been created. Rather than throwing more effort at this plugin, I'd prefer to help helend with enhancements to the TracSuposePlugin, or spend my time on other Trac plugin projects altogether.

I'd like to get some feedback and hear if anyone knows of a compelling reason to continue this project rather than moving to the TracSuposePlugin. Is there functionality in this plugin that doesn't exist in the TracSuposePlugin? I'm open to hearing all opinions and suggestions.

I'll leave these tickets open for about a week, but in all likelihood will close all of them and deprecate the plugin.

Thanks for your time,

  • Ryan

comment:8 Changed 7 years ago by anonymous

TracSupose looks very impressive, on a brief glance. Go for it.

comment:9 Changed 4 years ago by Ryan J Ollos

Owner: changed from Ryan J Ollos to anonymous

Modify Ticket

Change Properties
Set your email in Preferences
as new The owner will remain anonymous.

Add Comment

E-mail address and name can be saved in the Preferences.

Note: See TracTickets for help on using tickets.