id	summary	reporter	owner	description	type	status	priority	component	severity	resolution	keywords	cc	release
639	Large p4 depots kill virtual memory (patch included)	r.blum@gmx.net	lewisbaker	If you do an initial sync to a large depot (50,000 changelists), the perforce plugin will use up VM like crazy. (I stopped once I reached about 1.5 GB) \r\n\r\nSmall modification to the sync procedure solves that problem. (Basically, get changelists in chunks of 1000). Here's my local mod\r\n\r\n{{{\r\n#!python\r\n   # Override sync to precache data to make it run faster\r\n    def sync(self):\r\n        youngest_stored = self.repos.get_youngest_rev_in_cache(self.db)     \r\n        if youngest_stored is None:\r\n            youngest_stored = '0'\r\n\r\n        while youngest_stored != str(self.repos.youngest_rev):\r\n            # Need to cache all information for changes since the last\r\n            # sync operation.\r\n\r\n            youngest_to_get = self.repos.youngest_rev\r\n            if youngest_to_get > int(youngest_stored) + 1000:\r\n                youngest_to_get = int(youngest_stored) + 1000\r\n\r\n            # Obtain a list of changes since the last cache sync\r\n            from p4trac.repos import _P4ChangesOutputConsumer\r\n            output = _P4ChangesOutputConsumer(self.repos._repos)\r\n            self.repos._connection.run('changes', '-l', '-s', 'submitted',\r\n                                       '@>%s,%d' % ( youngest_stored, youngest_to_get ),\r\n                                       output=output)\r\n\r\n            if output.errors:\r\n                from p4trac.repos import PerforceError\r\n                raise PerforceError(output.errors)\r\n\r\n            changes = output.changes\r\n            changes.reverse()\r\n\r\n            # Perform the precaching of the file history for files in these\r\n            # changes.\r\n            self.repos._repos.precacheFileHistoryForChanges(changes)\r\n\r\n            youngest_stored=str(youngest_to_get)\r\n\r\n        # Call on to the default implementation now that we've cached\r\n        # enough information to make it run a bit faster.\r\n        CachedRepository.sync(self)\r\n}}}	enhancement	closed	normal	PerforcePlugin	normal	duplicate	needinfo		0.10