Caching of file paths and configs

[PugFish]

Caching the file paths found by Kohana::find_file() across requests can dramatically increase the performance of an application. Also caching the merged config files can give another performance boost (this also applies to i18n and L10n files as they behave the same as config files, but isn't implemented in my patch).

I've implemented some changes to the Kohana and Config core classes, these can be found in the attached patch file. It's not release worthy code, but it does work well enough to test with.

I have done some extensive testing with this code, the results are below. I measured the amount of requests a very simple application could output in 10 seconds, adding modules and configs to see how it handles them. For the "Configs" tests I created 10 config files in the application/config folder with 20 entries in each, and accessed an entry from each file so they would be loaded. For the "Configs in modules" tests these configs were copied into the empty modules so that Kohana would merge them. The rest should be self explanatory.

• No cache:
  • No modules:
    • No configs: 552
    • Configs: 485
  • 1 module:
    • No configs: 470
    • Configs: 382
    • Configs in module: 354
  • 2 modules:
    • No configs: 391
    • Configs: 330
    • Configs in modules: 322

• File path cache:
  • No modules:
    • No configs: 842
    • Configs: 736
  • 1 module:
    • No configs: 824
    • Configs: 750
    • Configs in module: 686
  • 2 modules:
    • No configs: 830
    • Configs: 754
    • Configs in modules: 657

• File path and config cache:
  • No modules:
    • No configs: 843
    • Configs: 813
  • 1 module:
    • No configs: 841
    • Configs: 816
    • Configs in module: 806
  • 2 modules:
    • No configs: 840
    • Configs: 810
    • Configs in modules: 806

As you can see without any caching the performance drops rapidly when you start adding even empty modules, the extra file system calls needed to search them are killer. When you add in extra config files this just gets worse. That's about 40% drop in throughput and it's not even doing anything yet.

When adding file path caching straight away there is a roughly 33% increase in throughput, and each extra module has no hit. When loading the configs however there is a 10% decrease for each extra module.

Enabling the file path AND config caching turns that 40% drop without caching into less than 5%, and is 250% faster overall!

[PugFish]

Fixed a bug in the patch where files of the same name but different extensions were cached wrong. Also updated with latest find_file changes.

[PugFish]

There were some concerns about whether very large configs would see a negative effect from caching due to the speed of serialising the large cache file. I changed those 10 config files from 20 entries to 5000 entries each (making the cache file 1.9mb in size) and reran the same test as before.

• 10 configs, 20 entries each, no modules (from above):
  • No cache: 485
  • File path and config cache: 813

• 10 configs, 5000 entries each, no modules:
  • No cache: 47
  • File path and config cache: 115

Even with an unusually large cache file it's still far faster, and the benefit gets even bigger (167% vs 244%)

[Shadowhand]

Fixed in r2714.

[Shadowhand]

Follow up in r2715.