Commit Graph

21 Commits

Author SHA1 Message Date
0fd85b76db automatic archive builder now functional and enabled.
- allocators.cpp: fix: make sure all pool fields are reset in
pool_free_all; correct matrix self-test
- lib: fix self-test
- lib_error: give "informational" return codes values > 100 (allows
multiplexing with percent indicator)
- archive: add clarification to afile_enum; use vfs_load when building
archive (fixes caching problem); add support for progressive archive
creation
- file: allow pp_set_dir's path to end in slash (simplifies other code);
add file_exists
- file_cache: add stats+sanity checks to allocator; improve ExtantBufMgr
find(); add ExactBufOracle to translate padded address returned by Cache
to what was actually allocated; relax infinite-loop check; fix
file_cache_reset
- file_io: refactor callback logic so it can be used by vfs (needed for
vfs_load)
- vfs: vfs_load now supports calling back (required for archive builder)
- vfs_mount: optimize path caching in afile_cb: due to atom_fn system,
string compare can be done via pointer compare
- vfs_optimizer: support auto-build of archive (progressive); improve
should-build-archive check (WIP)
- vfs_path: add util functions (also used in screenshot code)
- vfs_tree: simplify pathname storage
- wposix: mmap bugfix
- main: call progressive archive builder; skip reload when doing so
(documented)
- JSInterface_VFS+scriptglue: add archiveBuilderCancel
- ps/Util: use new vfs_path helper function for cleverer screenshot
naming (correctly handles some being deleted)

This was SVN commit r3578.
2006-03-01 22:31:11 +00:00
4a0baf529d rename free() to dealloc() to avoid need for #include "nommgr.h"
This was SVN commit r3557.
2006-02-23 18:58:16 +00:00
bc07b9ff0a file code v2 is now feature complete and works.
- adts: add basic benchmark to self-test; use fminf instead of MIN; fix
overly pessimistic invalidation in Landlord_Cached
- allocators: no longer complain if DynArray is full
- lib: warn if rand() min=max
- archive: bugfixes: plug leak, correctly handle 0-length files, carry
over flags when reading uncompressed file from archive
- file: expose file_get_sorted_dirents and implement file_enum in terms
of that
- file_cache: many bugfixes (especially related to CacheAllocator
freelist)
- vfs_optimizer: finish implementation of archive builder
- zip: correctly handle 0-length files
- tex: add tex_is_known_extension
- sysdep: add efficient fminf implementation (assumes P6+)

- config: update archive builder call site
- gamesetup: add call for delayed self tests
- entity: return JSVAL_VOID as per philip's suggestion

This was SVN commit r3553.
2006-02-23 07:55:32 +00:00
0a368a4fec massive update to adts: cache now implemented via templated policy classes.
Entity.h: fix minor warning

This was SVN commit r3539.
2006-02-19 18:51:00 +00:00
5081b38c43 debug: remove old prototype
file_cache: use new remove_least_valuable interface
zip: add dox

This was SVN commit r3514.
2006-02-15 03:03:17 +00:00
c4e3037e60 adts: add LL_OPT_* defines that enable optimizations (some only make sense if there are tons of files, and need to be able to disable them for thesis). realized this would be perfect application of policy template classes, which will replace this.
fix crashdumps: was failing to write out to file (underlying cause:
current directory no longer being set)
app_hooks: add get_log_dir; used by debug and wdbg_sym

minor improvements/documentation in archive+compression+file_cache+zip

main: remove dead ScEd code

This was SVN commit r3498.
2006-02-11 22:49:09 +00:00
21f8f8555e adts: remove_least_valuable safely handles empty cache
lib: add u32_hi/lo
archive: move archive builder logic here from vf_optimizer
file_cache: add flush() call - used when simulating cache
vfs_optimizer: WIP, not yet functional: file gatherer, 50% of TSP tour
generator

This was SVN commit r3486.
2006-02-09 05:59:33 +00:00
920795fd85 file_cache: add refcounting to ExtantBufMgr (required for correctness)
This was SVN commit r3473.
2006-02-04 23:32:01 +00:00
2f438a6889 cache_allocator bugfix: do not attempt to coalesce allocations that are at start/end of committed region. fixes self-test failure.
This was SVN commit r3466.
2006-02-03 21:44:09 +00:00
987a6b7d4e fixed wchar_t issues; now correctly separates utf16, CStrW and jschar.
NOTE: wchar_t should be defined as a built-in type; no more need to
define manually or change project settings.

adts: disambiguate iterator type; add operator- as required by VC8
std::equal() implementation
precompiled: squelch stupid warning (that informs us that array members
are now actually default-initialized as called for by C++)
file_cache: remove debug variable

This was SVN commit r3464.
2006-02-03 20:36:15 +00:00
1227cb2f6d adts: add RingBuf self-test. RingBuf: make entry#0 the 'head' and pop_front a no-op if empty.
allocators: allow pool_create at NLSO ctor time
lib: add rand() function based on libc rand and self-test. (this allows
easily returning a random number within a range, and avoids a number of
pitfalls - the naive rand()%range approach is baaad)
precompiled: add self_test.h, so that self tests won't be accidentally
disabled by forgetting to include the header

file_cache: add self-test; fixed 2 bugs that exposed (was incorrectly
splitting large blocks and not setting class_size bit properly). fix
weird corner case in BlockMgr (attempting to free LRU block but it is
locked).

file_io, wdbg_sym: tiny fixes

This was SVN commit r3461.
2006-02-03 04:02:53 +00:00
7862e79e84 adts: optimize landlord algorithm at advisor's request (only relevant if we have tons of files in cache)
trace: make note of IO size

xmlutils: fix old remnant of Handle vfs_load return type.

This was SVN commit r3456.
2006-02-02 04:11:07 +00:00
d11fcc8257 file_cache: bugfix in CacheAllocator
hook up file_stats; implemented stats_dump (gives somewhat detailed
report at exit)

This was SVN commit r3448.
2006-01-31 07:48:32 +00:00
06412a7751 allocators: bugfix (wasn't setting freelist to 0)
archive: afile_read bugfix (wasn't returning byte count)
byte_order: add to_leXX, implement read_leXX in terms of that, add
write_leXX (and also big-endian versions)
compression: support 0 length buffers + minor fixes
file: no longer support passing fn_len (that didn't work anyway due to
DynHashTbl find(const char*) interface); add FILE_DONT_SET_FN
optimization that prevents cluttering name cache.
file_cache: add logic to verify load/free/load/free sequence; add
FILE_LONG_LIVED flag to specify exceptions (e.g. XMB files that don't
free their buffer immediately)
vfs_mount/vfs_tree: bugfix (was mixing up portable and VFS paths)
vfs_optimizer: small fixes; creating Zip files now works
vfs_tree: add PathName that stores pointer to path and name component
for efficiency + convenience
zip: bugfixes (wasn't endian safe; incorrect handling of cmethod; forgot
to initialize ZipArchive members)

config: add -buildarchive flag
Xeromyces.cpp, FileUnpacker.cpp: add FILE_LONG_LIVED
mapreader: bump time estimate for ReadXML

This was SVN commit r3446.
2006-01-31 03:47:52 +00:00
ecbcc59fba bugfix: no longer make freed buffers readonly because we need to write boundary tags into them for coalescing
This was SVN commit r3432.
2006-01-28 23:02:26 +00:00
cda1e4c177 bitten by free() redirector macro again.. fixed
This was SVN commit r3430.
2006-01-28 22:30:28 +00:00
e07622b56a Cache: implement meat of landlord algorithm and add remove()
allocators: add freelist capability to Bucket; add provision for
variable XOR fixed size allocs
archive: re-tag file buffers if reading uncompressed from archive;
improve LFH fixup logic
file_cache: add cache line invalidation; lock down pages (readonly) when
IO finished
file_io: cleanup+docs; properly cut off at EOF without breaking
alignment.
file_stats: add seek accounting (WIP)
vfs_optimizer: also record file_buf_free in the trace. initial
implementation of archive builder (WIP)
zip: lfh_fixup now more efficient (does not involve buffer manager -
instead it grabs LFH from temp blocks)
tex: plug FileIOBuf leak. avoid writing to tex.hm because that is a
read-only file_buf.

This was SVN commit r3428.
2006-01-28 22:19:42 +00:00
3b4295a177 Linux build fixes
This was SVN commit r3427.
2006-01-28 22:04:09 +00:00
4a508c6b87 all lib/res/file headers included in file_internal.h
split stats into separate file; different call mechanism; add lots of
stats (WIP)

This was SVN commit r3424.
2006-01-25 07:21:45 +00:00
7838627cd2 - vfs_load now returns error code and takes FileIOBuf; that must be freed via file_buf_free. if Handle is needed, use mem_wrap.
- remove ScEd hacks and CFont et al macro rename
- fix accursed bug in VFS buffer management that was causing ReadFile to
fail without error (not allocating enough padding)
- vfs_tree: bugfix in tree_lookup
- waio: temporarily disable sector size determination (pending better
approach - need to determine if using DVD drive)

This was SVN commit r3421.
2006-01-24 08:16:29 +00:00
36cbf23c8e v2 of file/res code for thesis:
previous zip.* split up into archive, zip and compression. add O(1)
lookup via memento. add support for writing Zip archive. simplify IO by
having compression module allow queuing input buffers.

file: split up into file, file_Io, file_cache. add atom_fn that enables
string comparison via pointer compare. (stores all filenames/paths in
one place). add statistics gathering.
file_cache: landlord algorithm for caching files as well as LRU block
cache.
file_io: aio code that splits transfers into blocks revamped and made
into class.

vfs: filelist/trace code moved into new vfs_optimizer. add caching logic
to vfs_load.
vfs_optimizer: will generate zip archive (to replace current archive
builder script). not yet done
vfs_tree: major cleanup - now has nice base class rather than union
hackery.

This was SVN commit r3420.
2006-01-23 20:05:09 +00:00