Modify

Opened 7 years ago

Closed 7 years ago

#2523 closed defect (worksforme)

PerforcePlugin resync consumes all my memory. Any chance for an incremental resync?

Reported by: p4plugin.eddieparker@… Owned by: ttressieres
Priority: normal Component: PerforcePlugin
Severity: normal Keywords:
Cc: Trac Release: 0.10

Description

Hello,

I'm not sure if this is the correct avenue for mentioning this, but I'd appreciate if anyone could offer me advice, or a way to code thsi feature.

Basically, I want to take a look at this PerforcePlugin, but our repository is rather large (almost 300,000 changelists) and it appears to be blowing my box's memory limit (1.5 GB RAM).

The resync *starts off* alright, but once it gets to about changelist 120,000 the server simply page faults and starts shutting down processes.

Is there any way to incrementally resync, so as to not consume all my memory?

Thanks!

Attachments (0)

Change History (4)

comment:1 Changed 7 years ago by ttressieres

  • Status changed from new to assigned

Hello,

The only way to perform incremental resync is to do trac-admin resync the first time and when there is an error you have to perform a synchronize by viewing the "Timeline".

The Timeline view performs an incremental sync

comment:2 Changed 7 years ago by anonymous

Hello, and thanks for getting back to me.

Does this work in 0.10?

I just tried loading http://mywebserver/trac/timeline. It started syncing, and I let it go until ti consumed about 30% of my memory. I then killed the thread, but on reloading the timeline, it still starts syncing from the very beginning.

Perhaps killing the thread isn't the right way to test this, but I didn't want to wait for the time it takes for it to run out of memory (usually about a day).

Just curious if I'm doing something wrong.

Are you sure there's nowhere I could put in the code a 'save-your-cache' line, so I could resume syncing from where it last took off?

Thanks!

comment:3 Changed 7 years ago by anonymous

Oh, I'm also using sqlite as a db. Could that be the problem? (i.e. when it crashes the file doesn't close so the state doesn't get saved?)

comment:4 Changed 7 years ago by anonymous

  • Resolution set to worksforme
  • Status changed from assigned to closed

Ah. I looked at the code, I see how this is a problem.

I ended up editing site-packages/trac/versioncontrol/cache.py, under the 'sync' function: I just made it commit to the database every instance of the loop. That allowed incremental things to run.

Thanks for pointing me in the right direction!

Add Comment

Modify Ticket

Action
as closed The owner will remain ttressieres.
The resolution will be deleted. Next status will be 'reopened'.
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.