Modify

Opened 10 years ago

Closed 10 years ago

#2523 closed defect (worksforme)

PerforcePlugin resync consumes all my memory. Any chance for an incremental resync?

Reported by: p4plugin.eddieparker@… Owned by: ttressieres
Priority: normal Component: PerforcePlugin
Severity: normal Keywords:
Cc: Trac Release: 0.10

Description

Hello,

I'm not sure if this is the correct avenue for mentioning this, but I'd appreciate if anyone could offer me advice, or a way to code thsi feature.

Basically, I want to take a look at this PerforcePlugin, but our repository is rather large (almost 300,000 changelists) and it appears to be blowing my box's memory limit (1.5 GB RAM).

The resync *starts off* alright, but once it gets to about changelist 120,000 the server simply page faults and starts shutting down processes.

Is there any way to incrementally resync, so as to not consume all my memory?

Thanks!

Attachments (0)

Change History (4)

comment:1 Changed 10 years ago by ttressieres

Status: newassigned

Hello,

The only way to perform incremental resync is to do trac-admin resync the first time and when there is an error you have to perform a synchronize by viewing the "Timeline".

The Timeline view performs an incremental sync

comment:2 Changed 10 years ago by anonymous

Hello, and thanks for getting back to me.

Does this work in 0.10?

I just tried loading http://mywebserver/trac/timeline. It started syncing, and I let it go until ti consumed about 30% of my memory. I then killed the thread, but on reloading the timeline, it still starts syncing from the very beginning.

Perhaps killing the thread isn't the right way to test this, but I didn't want to wait for the time it takes for it to run out of memory (usually about a day).

Just curious if I'm doing something wrong.

Are you sure there's nowhere I could put in the code a 'save-your-cache' line, so I could resume syncing from where it last took off?

Thanks!

comment:3 Changed 10 years ago by anonymous

Oh, I'm also using sqlite as a db. Could that be the problem? (i.e. when it crashes the file doesn't close so the state doesn't get saved?)

comment:4 Changed 10 years ago by anonymous

Resolution: worksforme
Status: assignedclosed

Ah. I looked at the code, I see how this is a problem.

I ended up editing site-packages/trac/versioncontrol/cache.py, under the 'sync' function: I just made it commit to the database every instance of the loop. That allowed incremental things to run.

Thanks for pointing me in the right direction!

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The owner will remain ttressieres.
The resolution will be deleted.

Add Comment


E-mail address and name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.