This is essentially the same ticket that I opened in #9764 for the QuietPlugin.
I've done some work over in #7617 with help from hasienda to improve the page load times when using the DateFieldPlugin. We replaced a virtual script (template) with a script and used add_script_data that is available now in Trac 0.12 to pass data to the script. The result is (and I barely know what I'm talking about here), the js file is under chrome and is treated as a cacheable static resource, resulting in improved page load times. I put a lot of info in #7617, documenting the improvements that I saw.
I'm thinking the same is probably possible here, though I haven't really looked at your source code much to see how you were doing things.
