Ability to to perform re-index with a quota on time / memory consumption

[Patrick Martin]

I don't think this is an easy one, but the time and resources required to re-index a large number of changes are quite significant.

Is it possible for a re-index to index a configurable number of files on each invocation? This would leave the repository search out of date, but release the server to serve the requests of other users. Susbsequent re-indexes could fill in the gap.

I'm not sure how valuable this would be if #365 were implemented, as one huge cost to re-indexing (tagging and branching) could be radically reduced. Nonetheless, repository search is transparently such a great feature is would be a shame if it became a victim of its own success when people with large-ish source trees (>1K files) have a go at using it.

[Alec Thomas]

The massive consumption of memory during indexing is a definite issue. In the near future I will be porting RepoSearchPlugin to use the ​pyndexter indexer abstraction layer. After this is complete I will probably rewrite the default indexer (essentially what is currently used in this plugin) as a C extension. These two changes should alleviate most performance related problems, particularly if #365 is implemented.

[Patrick Martin]

This sounds like great stuff.

Can't wait.

[Alec Thomas]

(In [1881]) Not for the faint of heart, but if anybody wants to test the pyndexter branch it would be most useful to get some feedback.

You will need the refactoring branch of pyndexter, available from ​here.

More info in the README.

References #362, #371, #385 and #388.

[Ryan J Ollos]

Reassigning ticket after changing maintainer of plugin.

[Ryan J Ollos]

Related to #385 (which is probably what is meant when referring to #365).

[Ryan J Ollos]

Hello,

I took over maintainership of this plugin from athomas some time ago. There is a significant amount of work to do on this plugin, and I don't foresee having the time to do it all.

helend has written the TracSuposePlugin, which seems like a much better solution. Rather than writing the repository search functionality from scratch, a Trac interface to an existing repository search tool has been created. Rather than throwing more effort at this plugin, I'd prefer to help helend with enhancements to the TracSuposePlugin, or spend my time on other Trac plugin projects altogether.

I'd like to get some feedback and hear if anyone knows of a compelling reason to continue this project rather than moving to the TracSuposePlugin. Is there functionality in this plugin that doesn't exist in the TracSuposePlugin? I'm open to hearing all opinions and suggestions.

I'll leave these tickets open for about a week, but in all likelihood will close all of them and deprecate the plugin.

Thanks for your time,

• Ryan

[anonymous]

TracSupose looks very impressive, on a brief glance. Go for it.