PerforcePlugin resync consumes all my memory. Any chance for an incremental resync?

[p4plugin.eddieparker@…]

Hello,

I'm not sure if this is the correct avenue for mentioning this, but I'd appreciate if anyone could offer me advice, or a way to code thsi feature.

Basically, I want to take a look at this PerforcePlugin, but our repository is rather large (almost 300,000 changelists) and it appears to be blowing my box's memory limit (1.5 GB RAM).

The resync *starts off* alright, but once it gets to about changelist 120,000 the server simply page faults and starts shutting down processes.

Is there any way to incrementally resync, so as to not consume all my memory?

Thanks!

[ttressieres]

Hello,

The only way to perform incremental resync is to do trac-admin resync the first time and when there is an error you have to perform a synchronize by viewing the "Timeline".

The Timeline view performs an incremental sync

[anonymous]

Hello, and thanks for getting back to me.

Does this work in 0.10?

I just tried loading ​http://mywebserver/trac/timeline. It started syncing, and I let it go until ti consumed about 30% of my memory. I then killed the thread, but on reloading the timeline, it still starts syncing from the very beginning.

Perhaps killing the thread isn't the right way to test this, but I didn't want to wait for the time it takes for it to run out of memory (usually about a day).

Just curious if I'm doing something wrong.

Are you sure there's nowhere I could put in the code a 'save-your-cache' line, so I could resume syncing from where it last took off?

Thanks!

[anonymous]

Oh, I'm also using sqlite as a db. Could that be the problem? (i.e. when it crashes the file doesn't close so the state doesn't get saved?)

[anonymous]

Ah. I looked at the code, I see how this is a problem.

I ended up editing site-packages/trac/versioncontrol/cache.py, under the 'sync' function: I just made it commit to the database every instance of the loop. That allowed incremental things to run.

Thanks for pointing me in the right direction!