1) unify open/io interfaces: no longer distinguish between FileIo /
AFileIo. instead, one FileIo struct is always used; the
provider-specific data is embedded within it. this avoids the need for a
"base class", which is hard to access (f.fc.flags is ugly)
2) rename x_* redirector to xfile_* and move out of vfs_mount into
separate file
3) xfile is now implemented via "virtual functions" (avoids a lot of
boilerplate code)
4) xfile was too closely tied to vfs_mount (used internal fields);
fixed.
remaining lib/res files: made all flag parameters uint (not int) for
consistency
This was SVN commit r3776.
- color: the sse codepath is now detected and activated from within
color.cpp, which avoids making ia32 dependent on the color header. it's
called from gamesetup!InitRenderer.
- move VFSUtil::EnumDirEnts to vfs.cpp!vfs_dir_enum - allows it to be
used from within lib/ without dependency on ps (annoying for other
projects)
This was SVN commit r3764.
snd_mgr: replace al_check("funcname") with AL_CHECK macro; allow
disabling WIN_LOADLIBRARY_HACK.
wcpu: move sys_on_each_cpu here from wsysdep
split up wdetect into wgfx, wsnd, wdll_ver
This was SVN commit r3748.
- split up detect.cpp into snd, cpu, gfx (headers were already split)
- rename detect functions for more clarity (e.g. get_cur_vmode ->
gfx_get_video_mode)
- rename lib/res/sound/snd to snd_mgr to avoid conflict with
sysdep/snd.cpp
- waio: set errno in aio_rw (addresses bug #73)
This was SVN commit r3745.
adts: use SAFE_FREE
config: add CONFIG_FINAL; use it in
compression,vfs_optimizer,scriptablecomplex
lib: add discussion of unreachable code avoidance; revise UNREACHABLE
and add NODEFAULT
ogl_tex, tex_dds: remove some unnecessary UNREACHABLE
sysdep: move compiler-specific implementation of UNREACHABLE here.
use NODEFAULT in snd, wdbg, wposix, wsdl
This was SVN commit r3739.
FILE_CACHE_BLOCK flag used more safely (copied in afile_read)
add file_sector_size (set from sysdep code)
documented and organized FileFlags
moved publically visible file_buf* decls to file.h; add FileBufFlags
file_buf_get: moved to file_io and renamed because it is not a part of
the file_buf manager itself
FILE_BLOCK_SIZE no longer exposed to users - it's an implementation
detail. for alignment purposes, you should use file_sector_size (less
wasteful)
add page_aligned allocator (mmap)
write buffers now allocated via file_buf_alloc(FB_FROM_HEAP) (avoids
wasting cache space)
append trace runs into the trace file (-> better input data for archive
ordering)
document lib_errors.h
tex_load: add provision for file_flags, used by ogl_tex_load to prevent
caching file data
simulation, entitymanager: add perf measurements
This was SVN commit r3725.
* replace all "return ERR_*" with WARN_RETURN(ERR_*) - makes sure
function failures are noticed *at the cause*, not later
* LibError_from_* now take bool warn_if_failed param
* replace more debug_warns with dedicated error codes (allows localized
error reports and doesn't spam the EXE with strings)
This was SVN commit r3722.
adts: whip together inefficient LRU cache manager (mostly for comparing
vs. Landlord)
file_cache: fix self-test (now correctly deals with small cache sizes)
file_stats: more stats for file cache
trace, vfs: bugfix: ignore writes recorded by the trace
This was SVN commit r3684.
adts: slight improvements
allocators: change POOL_VARIABLE_ALLOCS to -1 for safety; fix invalid
assumptions of it being 0
lib: more expressive param names
file: add file_get_random_name for randomized trace generation; allow
file_cache.h to declare externally visible file_buf_alloc API
file_cache: zero-length file bugfix; also add several flags to the API
routines allowing statistics gathering to be disabled (avoids distorting
stats due to e.g. trace_entry_causes_io cache simulation)
file_stats: add archive builder info; separate IO timing and seek
accounting (-> correct results)
trace: add random generation capability
vfs: made responsible for cache hit/miss accounting (fixes cache miss
rate display)
This was SVN commit r3681.
file: change block size to 32kb (=> yields highest throughput)
archive, file_io: add additional block cache forwarding path when
reading uncompressed data from archive (avoids repetitive IOs)
file_io, file_stats: fix inaccurate and incorrect IO throughput
calculation
file_stats: refactor some calculations to avoid division by 0
This was SVN commit r3665.
vfs_mount: only notify archive builder of files that are archivable
vfs_optimizer: fix should-rebuild logic; make 2 criteria optional and
disable in final release for convenience
This was SVN commit r3651.
vfs: remove second call to tree_init (now done by vfs_mount)
vfs_tree: fix memory leak due to not destroying root node. commented and
armor-plated things while at it.
This was SVN commit r3647.
- lib, sysdep.h: move DIR_SEP to sysdep.h
- archive, compression: minor touchups
- zip: large refactor: all accesses to little-endian structs are now
done by helper functions (fixes bug: wasn't endian swapping ECDR). also
detect unfinished zip files (for VFS mount to ignore)
- tex_dds: fix comment
This was SVN commit r3634.
file_cache: add warning to cache_allocator to catch incorrect usage /
bugs
vfs_optimizer.cpp: tweak miniarchive threshold (currently have config
files in there; don't want those to trigger rebuild every time)
zip: clarification
tex_dds: we now bother checking the DDPIXELFORMAT mask bits because
uncompressed DDS files may be stored in any pixel component order, but
we require RGB for efficiency.
wsdl: fix failure to update; was constantly queuing events if mouse was
out of window
This was SVN commit r3627.
vfs: add VFS_MOUNT_ARCHIVABLE flag; allow archive builder to only
include files from mount points with this set.
VFSUtil: make EnumDirEnts more efficient while at it (uses PathPackage
and filename storage)
This was SVN commit r3618.
trace: split code out of vfs_optimizer (was getting too large)
vfs_mount: expose API to unmount all (needed to be able to delete
archives after building)
vfs_optimizer: safely handle empty connection/node lists;
refactored+documented
vfs_tree: eliminate 'file in root dir' warning (can happen easily
enough)
rest: fixes/minor tweaks
This was SVN commit r3610.
- file_cache: add checker for allocations (catches bugs in debug mode);
fix bug that freed buffers even thought they were on extant list.
- vfs_mount: bugfix: was passing P_path to vfs_opt_notify when it should
be V_path
- vfs_optimizer: big refactor; fix bugs in TourBuilder causing some
files to be omitted from archive (thus forcing unnecessary archive
rebuild - because #files not in archive was too big)
This was SVN commit r3602.
- adts: Cache.remove_least_valuable was returning false prematurely even
though there were still some items pending eviction; that meant they
remained in cache
- file_cache: wasn't clearing exact_buf_oracle on reset. also made
make_read_only pass correct exact_buf rather than just rounding down.
This was SVN commit r3589.
- compression: now use "fastest" compression mode and more memory unless
in FINAL build. this greatly speeds up archive generation at expense of
1.5% size increase (fine during development). done because profiling
shows ZLib accounts for 78% of CPU; practically nothing else shows up,
which shows effectiveness of compressing in parallel with reading from
disk.
- file_cache: documented; fixed several potential issues with exact_buf
when invalidating files.
This was SVN commit r3586.
everything else: bugfix: buffers added to extant list via
file_buf_retrieve (i.e. moved from cache to extant list) didn't have
their long_lived flag carried over; this caused a "not freed
immediately" warning.
This was SVN commit r3580.
- allocators.cpp: fix: make sure all pool fields are reset in
pool_free_all; correct matrix self-test
- lib: fix self-test
- lib_error: give "informational" return codes values > 100 (allows
multiplexing with percent indicator)
- archive: add clarification to afile_enum; use vfs_load when building
archive (fixes caching problem); add support for progressive archive
creation
- file: allow pp_set_dir's path to end in slash (simplifies other code);
add file_exists
- file_cache: add stats+sanity checks to allocator; improve ExtantBufMgr
find(); add ExactBufOracle to translate padded address returned by Cache
to what was actually allocated; relax infinite-loop check; fix
file_cache_reset
- file_io: refactor callback logic so it can be used by vfs (needed for
vfs_load)
- vfs: vfs_load now supports calling back (required for archive builder)
- vfs_mount: optimize path caching in afile_cb: due to atom_fn system,
string compare can be done via pointer compare
- vfs_optimizer: support auto-build of archive (progressive); improve
should-build-archive check (WIP)
- vfs_path: add util functions (also used in screenshot code)
- vfs_tree: simplify pathname storage
- wposix: mmap bugfix
- main: call progressive archive builder; skip reload when doing so
(documented)
- JSInterface_VFS+scriptglue: add archiveBuilderCancel
- ps/Util: use new vfs_path helper function for cleverer screenshot
naming (correctly handles some being deleted)
This was SVN commit r3578.
- adts: add basic benchmark to self-test; use fminf instead of MIN; fix
overly pessimistic invalidation in Landlord_Cached
- allocators: no longer complain if DynArray is full
- lib: warn if rand() min=max
- archive: bugfixes: plug leak, correctly handle 0-length files, carry
over flags when reading uncompressed file from archive
- file: expose file_get_sorted_dirents and implement file_enum in terms
of that
- file_cache: many bugfixes (especially related to CacheAllocator
freelist)
- vfs_optimizer: finish implementation of archive builder
- zip: correctly handle 0-length files
- tex: add tex_is_known_extension
- sysdep: add efficient fminf implementation (assumes P6+)
- config: update archive builder call site
- gamesetup: add call for delayed self tests
- entity: return JSVAL_VOID as per philip's suggestion
This was SVN commit r3553.
fix crashdumps: was failing to write out to file (underlying cause:
current directory no longer being set)
app_hooks: add get_log_dir; used by debug and wdbg_sym
minor improvements/documentation in archive+compression+file_cache+zip
main: remove dead ScEd code
This was SVN commit r3498.
lib: add u32_hi/lo
archive: move archive builder logic here from vf_optimizer
file_cache: add flush() call - used when simulating cache
vfs_optimizer: WIP, not yet functional: file gatherer, 50% of TSP tour
generator
This was SVN commit r3486.
NOTE: wchar_t should be defined as a built-in type; no more need to
define manually or change project settings.
adts: disambiguate iterator type; add operator- as required by VC8
std::equal() implementation
precompiled: squelch stupid warning (that informs us that array members
are now actually default-initialized as called for by C++)
file_cache: remove debug variable
This was SVN commit r3464.
allocators: allow pool_create at NLSO ctor time
lib: add rand() function based on libc rand and self-test. (this allows
easily returning a random number within a range, and avoids a number of
pitfalls - the naive rand()%range approach is baaad)
precompiled: add self_test.h, so that self tests won't be accidentally
disabled by forgetting to include the header
file_cache: add self-test; fixed 2 bugs that exposed (was incorrectly
splitting large blocks and not setting class_size bit properly). fix
weird corner case in BlockMgr (attempting to free LRU block but it is
locked).
file_io, wdbg_sym: tiny fixes
This was SVN commit r3461.
archive: afile_read bugfix (wasn't returning byte count)
byte_order: add to_leXX, implement read_leXX in terms of that, add
write_leXX (and also big-endian versions)
compression: support 0 length buffers + minor fixes
file: no longer support passing fn_len (that didn't work anyway due to
DynHashTbl find(const char*) interface); add FILE_DONT_SET_FN
optimization that prevents cluttering name cache.
file_cache: add logic to verify load/free/load/free sequence; add
FILE_LONG_LIVED flag to specify exceptions (e.g. XMB files that don't
free their buffer immediately)
vfs_mount/vfs_tree: bugfix (was mixing up portable and VFS paths)
vfs_optimizer: small fixes; creating Zip files now works
vfs_tree: add PathName that stores pointer to path and name component
for efficiency + convenience
zip: bugfixes (wasn't endian safe; incorrect handling of cmethod; forgot
to initialize ZipArchive members)
config: add -buildarchive flag
Xeromyces.cpp, FileUnpacker.cpp: add FILE_LONG_LIVED
mapreader: bump time estimate for ReadXML
This was SVN commit r3446.
allocators: add freelist capability to Bucket; add provision for
variable XOR fixed size allocs
archive: re-tag file buffers if reading uncompressed from archive;
improve LFH fixup logic
file_cache: add cache line invalidation; lock down pages (readonly) when
IO finished
file_io: cleanup+docs; properly cut off at EOF without breaking
alignment.
file_stats: add seek accounting (WIP)
vfs_optimizer: also record file_buf_free in the trace. initial
implementation of archive builder (WIP)
zip: lfh_fixup now more efficient (does not involve buffer manager -
instead it grabs LFH from temp blocks)
tex: plug FileIOBuf leak. avoid writing to tex.hm because that is a
read-only file_buf.
This was SVN commit r3428.
- made file_io_wait buffer non-const (less const_casting around -
buffers are really non-const. besides which, inadvertent use will be
caught via MMU)
- zip: disambiguate abs() param
This was SVN commit r3426.
- remove ScEd hacks and CFont et al macro rename
- fix accursed bug in VFS buffer management that was causing ReadFile to
fail without error (not allocating enough padding)
- vfs_tree: bugfix in tree_lookup
- waio: temporarily disable sector size determination (pending better
approach - need to determine if using DVD drive)
This was SVN commit r3421.
previous zip.* split up into archive, zip and compression. add O(1)
lookup via memento. add support for writing Zip archive. simplify IO by
having compression module allow queuing input buffers.
file: split up into file, file_Io, file_cache. add atom_fn that enables
string comparison via pointer compare. (stores all filenames/paths in
one place). add statistics gathering.
file_cache: landlord algorithm for caching files as well as LRU block
cache.
file_io: aio code that splits transfers into blocks revamped and made
into class.
vfs: filelist/trace code moved into new vfs_optimizer. add caching logic
to vfs_load.
vfs_optimizer: will generate zip archive (to replace current archive
builder script). not yet done
vfs_tree: major cleanup - now has nice base class rather than union
hackery.
This was SVN commit r3420.
byte_order: add FOURCC_BE and _LE versions (needed for zip)
ogl: quick hack: prevent crash on laptop by providing a
pglDrawRangeElementsEXT stub
file, h_mgr: update pool_alloc call site
tex: fix incorrect indexing/stride for mipmap generation
vfs_mount: disable archives (zip code is WIP)
zip: heavy WIP. add support for writing archives (needed for thesis);
revised Zip read code. sped up archive open. further major changes
pending.
wposix: moved allocator code (see above)
This was SVN commit r3298.
- add translators from errno and GetLastError to LibError
- clarified return values of callbacks (they must return
INFO_CB_CONTINUE to continue)
- this exposed a few bugs in error handling chains (returning incorrect
values); also reduced say-nothing instances of return -1.
- move CHECK_ERR etc. macros to lib_error
This was SVN commit r3229.
- config: add CONFIG_TRACE
- display_error_impl->sys_display_error
- cleaned up sysdep; add sys_ prefix everywhere and document everything
- add to vfs_load dox
- cursor: sys_cursor_load -> sys_cursor_create. sysdep code is no longer
dependent on tex; instead of calling tex_load, the caller passes a BGRA
texture in.
memcpy: huge kick in the pants for accompanying paper; now even faster.
- on_each_cpu -> sys_on_each_cpu (removed manager function also)
- wsdl: explain PeekMessage CPU usage issue
This was SVN commit r3203.
vfs_tree: move bucket allocator to allocators.cpp
allocators: add matrix self-test
zip: remove redundant timing code and add CHECK_ZFILE everywhere
ogl_tex: implement app hook for overriding gl upload caps
tex: add CHECK_TEX everywhere
TextureManager: only call LoadTextures once (instead of for each file
type)
debug: add hook for translating error messages; add filtering mechanism
to only show interesting debug_printfs (WIP). restructured code:
debug_puts is the sysdep part, while debug_printf does filtering and
then calls that
This was SVN commit r2990.
Fix: Unix symbol resolver ignores NULL parameters like Windows resolver
instead of crashing.
Fix: Bugs and crashes in the input system, caused by wrong use of
SDL_BUTTON_* constants.
This was SVN commit r2984.
vfs: move public functions into vfs.cpp to avoid confusion (except
vfs_mount in vfs_mount.cpp, but that's obvious)
h_mgr: fix bug in filename display causing dir names to be truncated
main/gamesetup: replace 2 bools with flags; remove Atlas trampoline
functions (clutters up main.cpp)
atlas: use GameSetup.h instead of extern ..
This was SVN commit r2983.
vfs: now always check filter in VDir (Not only in debug mode)
vfs_tree: free VFS nodes to avoid them appearing as "leaks". bugfix in
bucket allocator - wasn't coping with exactly filled buckets correctly
all Handle users: add to_string method that writes the interesting parts
of a resource's user data to string.
h_mgr: show this information when a handle is closed
ogl_tex: add ogl_tex_find
add 2 timer clients
mem: now record address of function that allocated memory
GameSetup: fix shutdown order
Renderer: allow freeing individual alpha map textures (we cache the
composite)
This was SVN commit r2981.
ia32, sysdep: add rint()
minimap: saw TODO on GetMapSpaceCoords and sped it up by 15% using rintf
and precalculating scaleX
timer: add indication of how often something was billed (helps measure
stuff called an indeterminate number of times)
This was SVN commit r2973.
-move tex_codec stuff out of tex.cpp and into tex_codec.cpp
-tex_codec: use linked list of codecs instead of array (simplifies code
and removes limit)
-ogl_tex: add override mechanism for s3tc/automipmapgen extension detect
-tired of keeping of debug_warn text in sync with function name; now use
__func__ everywhere (and emulate that with __FUNCTION__ on MSC)
-add ONCE_NOT (opposite of ONCE)
-fix 3!! stupid bugs in ia32_control87, its caller and the header that
were canceling each other out. float exceptions now enabled except for
"inexact result" (too common). (update: nicolai had already fixed 2 of
these)
This was SVN commit r2964.
- enable floating point exceptions (helps catch bugs)
file.cpp: improved error checking in dir_open/dir_next_ent code
vfs.h: expand docs; add note on patching + SCM
main: use spf (1/fps) to derive TimeSinceLastFrame (makes for smoother
updates)
timer: rename SUM_TIMER to TIMER_ACCRUE. interface change allows
unlimited number of timers (instead of static limit as before).
documented everything
This was SVN commit r2958.
GUI/input code: moved g_active, g_keys etc. into ps/Globals; it has its
own input handler as well
all users of fopen(): add file_make_full_native_path to get correct path
even though current dir = system/
This was SVN commit r2954.
TextureEntry: add SUM_TIMER around base color calculation
Zip: add SUM_TIMER around inflate/memcpy
main: re-removed g_MaxZoomHeight - this doesn't belong in the message
loop. it resides in GameView.cpp
This was SVN commit r2945.
config: proper detect for clock_gettime
file: file_rel_chdir -> file_set_root_dir; it no longer changes current
directory; instead, root dir is prepended to all paths (not that bad
perf-wise)
netlog: avoid a CRT assert if log file open failed. now that current dir
is no longer being set, we need to think about what will happen to the
logs - they either need to use file_make_full_native_path, or use the
entire file.cpp interface instead of fopen etc.
This was SVN commit r2936.
ogl_tex: bind() implementation much simplified; also fix bug that set
TMU correctly when disabling texturing
wsdl: fix stdout.txt redirect; now works regardless of current
directory. also no longer disable file buffering except when in full
debug mode (because it's so slow). also add minimize button to caption
bar in windowed mode.
This was SVN commit r2919.
dyn_array.h -> allocators.h
ogl_tex: reenable performance warning
tex: comments
h_mgr: debug_printf if long filename is passed in (so we notice)
wsdl: squelch DestroyWindow warning. will look into the cause.
sysdep: slightly simplify "restrict" determination
This was SVN commit r2916.
file: add timer
h_mgr bugfix: h_force_free wasn't clearing refcnt
mem: add note on shutdown order
snd: bugfixes: VSrc_validate was overzealous (al_src == 0 is allowed);
sd->is_valid wasn't being set -> leaked AL buffers.
wsdl: remove overzealous ReleaseDC check (false alarm)
gamesetup: clarify (resource) shutdown order
This was SVN commit r2913.
- add automatically called validate function
- make sure dtors correctly handle partial reload failures
all file code: IO routines renamed to io_* from *_io
h_mgr: add suballocator for filenames; some reorganization+cleanup;
better example code and added some notes to docs
snd: reorg to move resource methods next to struct
This was SVN commit r2907.
dyn_array: add pool allocator - see comments.
file(+gamesetup): add suballocator for aiocb (requires shutdown); revise
validator; rename file_*_io to file_io_*
This was SVN commit r2906.
fix 2 bugs in tex / ogl_tex reported by philip (related to caching) -
thanks!
tex: documented header and improved implementation. add tex_transform_to
for convenience. change tex_write to require going through tex_wrap.
This was SVN commit r2837.
fix UNREACHABLE def on unix;
string_s: use HAVE_STRING_S;
wdbg_sym: fix option set code; explain why search path isn't needed
WriteScreenshot: make sure there's enough mem; use Handle to free mem
(more efficient+less error prone)
This was SVN commit r2779.
lib: add bits() function (actually moved it from zip.cpp)
ia32: start moving code out to external asm file (will use NASM in
pre-build step; avoids needing MSC and GCC-specific inline code). move
memcpy test code out to other file
This was SVN commit r2706.
file: expose block size constant; always round user transfers up
tex: transform() now receives what to change, not what to achieve.
tex*: encoding works with DynArray+VFS backend
GameSetup: free g_BuildingPlacer to avoid mem leak.
fixed some warnings+bugs.
This was SVN commit r2679.
- mem.h: export mem_assign (needed to wrap a mem buffer)
- use stricmp in some file extension comparisons
- wposix: properly sets last-error code when opendir fails
- also: misc comments
This was SVN commit r2650.
- wposix.cpp: initial support for MEM_RESERVE and MEM_COMMIT semantics
in mmap
- cstr: removed no longer necessary serialization header
- xmlutils: wrap new() calls in nommgr/mmgr; allows tracking other
allocs in this file.
- add u64_from_u32
- various minor comments/improvements.
This was SVN commit r2604.
