Modify

Opened 8 years ago

Last modified 7 weeks ago

#388 new enhancement

Ability to to perform re-index with a quota on time / memory consumption

Reported by: patrickm Owned by:
Priority: normal Component: RepoSearchPlugin
Severity: normal Keywords:
Cc: Trac Release: 0.9

Description

I don't think this is an easy one, but the time and resources required to re-index a large number of changes are quite significant.

Is it possible for a re-index to index a configurable number of files on each invocation? This would leave the repository search out of date, but release the server to serve the requests of other users. Susbsequent re-indexes could fill in the gap.

I'm not sure how valuable this would be if #365 were implemented, as one huge cost to re-indexing (tagging and branching) could be radically reduced.
Nonetheless, repository search is transparently such a great feature is would be a shame if it became a victim of its own success when people with large-ish source trees (>1K files) have a go at using it.

Attachments (0)

Change History (9)

comment:1 Changed 8 years ago by athomas

  • Status changed from new to assigned

The massive consumption of memory during indexing is a definite issue. In the near future I will be porting RepoSearchPlugin to use the pyndexter indexer abstraction layer. After this is complete I will probably rewrite the default indexer (essentially what is currently used in this plugin) as a C extension. These two changes should alleviate most performance related problems, particularly if #365 is implemented.

comment:2 Changed 8 years ago by patrickm

This sounds like great stuff.

Can't wait.

comment:3 Changed 8 years ago by athomas

(In [1881]) Not for the faint of heart, but if anybody wants to test the pyndexter branch
it would be most useful to get some feedback.

You will need the refactoring branch of pyndexter, available from
here.

More info in the README.

References #362, #371, #385 and #388.

comment:4 Changed 5 years ago by rjollos

Reassigning ticket after changing maintainer of plugin.

comment:5 Changed 5 years ago by rjollos

  • Owner changed from athomas to rjollos
  • Status changed from assigned to new

comment:6 Changed 5 years ago by rjollos

  • Summary changed from ability to to perform re-index with a quota of time / memory consumption to Ability to to perform re-index with a quota on time / memory consumption

Related to #385 (which is probably what is meant when referring to #365).

comment:7 Changed 3 years ago by rjollos

Hello,

I took over maintainership of this plugin from athomas some time ago. There is a significant amount of work to do on this plugin, and I don't foresee having the time to do it all.

helend has written the TracSuposePlugin, which seems like a much better solution. Rather than writing the repository search functionality from scratch, a Trac interface to an existing repository search tool has been created. Rather than throwing more effort at this plugin, I'd prefer to help helend with enhancements to the TracSuposePlugin, or spend my time on other Trac plugin projects altogether.

I'd like to get some feedback and hear if anyone knows of a compelling reason to continue this project rather than moving to the TracSuposePlugin. Is there functionality in this plugin that doesn't exist in the TracSuposePlugin? I'm open to hearing all opinions and suggestions.

I'll leave these tickets open for about a week, but in all likelihood will close all of them and deprecate the plugin.

Thanks for your time,

  • Ryan

comment:8 Changed 3 years ago by anonymous

TracSupose looks very impressive, on a brief glance.
Go for it.

comment:9 Changed 7 weeks ago by rjollos

  • Owner rjollos deleted

Add Comment

Modify Ticket

Action
as new .
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.