Signal and setup events correctly
Eliminate possible races
Use a single event
Mark volatiles and reduce scope of waithandles
Common handler
100ms -> 50ms
* Salieri: Add blacklist system and blacklist shaders using bindless
Currently the shader cache doesn't have the right format to support
bindless textures correctly and may cache shaders that it cannot rebuild
after host invalidation.
This PR address the issue by blacklisting shaders using bindless
textures.
THis also support detection of already cached broken shader and handle removal
of those.
* Move to a feature flags design to avoid intrusive changes in the translator
This remove the auto correct behaviour
* Reduce diff on TranslationFlags
* Reduce comma on last entry of TranslationFlags
* Fix inverted logic and remove leftovers
* remove debug edits oops
* Improve linear texture compatibility rules
Fixes an issue where small or width-aligned (rather than byte aligned) textures would fail to create a view of existing data. Creates a copy dependency as size change may be risky.
* Minor cleanup
* Remove Size Change for Copy Depenedencies
The copy to the target (potentially different sized) texture can properly deal with cropping by itself.
* Move StrideAlignment and GobAlignment into Constants
* Improve Buffer Textures and flush Image Stores
Fixes a number of issues with buffer textures:
- Reworked Buffer Textures to create their buffers in the TextureManager, then bind them with the BufferManager later.
- Fixes an issue where a buffer texture's buffer could be invalidated after it is bound, but before use.
- Fixed width unpacking for large buffer textures. The width is now 32-bit rather than 16.
- Force buffer textures to be rebound whenever any buffer is created, as using the handle id wasn't reliable, and the cost of binding isn't too high.
Fixes vertex explosions and flickering animations in UE4 games.
* Set ImageStore flag... for ImageStore.
* Check the offset and size.
* Improve handling for unmapped GPU resources
- Fixed a memory tracking bug that would set protection on empty PTEs
- When a texture's memory is (partially) unmapped, all pool references are forcibly removed and the texture must be rediscovered to draw with it. This will also force the texture discovery to always compare the texture's range for a match.
- RegionHandles now know if they are unmapped, and automatically unset their dirty flag when unmapped.
- Partial texture sync now loads only the region of texture that has been modified. Unmapped memory tracking handles cause dirty flags for a texture group handle to be ignored.
This greatly improves the emulator's stability for newer UE4 games.
* Address feedback, fix MultiRange slice
Fixed an issue where the size of the multi-range slice would be miscalculated.
* Update Ryujinx.Memory/Range/MultiRange.cs (feedback)
Co-authored-by: Mary <thog@protonmail.com>
Co-authored-by: Mary <thog@protonmail.com>
* Initial implementation (3d tex mips broken)
This works rather well for most games, just need to fix 3d texture mips.
* Cleanup
* Address feedback
* Copy Dependencies and various other fixes
* Fix layer/level offset for copy from view<->view.
* Remove dirty flag from dependency
The dirty flag behaviour is not needed - DeferredCopy is all we need.
* Fix tracking mip slices.
* Propagate granularity (fix astral chain)
* Address Feedback pt 1
* Save slice sizes as part of SizeInfo
* Fix nits
* Fix disposing multiple dependencies causing a crash
This list is obviously modified when removing dependencies, so create a copy of it.
* Initial implementation of buffer flush (VERY WIP)
* Host shaders need to be rebuilt for the SSBO write flag.
* New approach with reserved regions and gl sync
* Fix a ton of buffer issues.
* Remove unused buffer unmapped behaviour
* Revert "Remove unused buffer unmapped behaviour"
This reverts commit f1700e52fb8760180ac5e0987a07d409d1e70ece.
* Delete modified ranges on unmap
Fixes potential crashes in Super Smash Bros, where a previously modified range could lie on either side of an unmap.
* Cache some more delegates.
* Dispose Sync on Close
* Also create host sync for GPFifo syncpoint increment.
* Copy buffer optimization, add docs
* Fix race condition with OpenGL Sync
* Enable read tracking on CommandBuffer, insert syncpoint on WaitForIdle
* Performance: Only flush individual pages of SSBO at a time
This avoids flushing large amounts of data when only a small amount is actually used.
* Signal Modified rather than flushing after clear
* Fix some docs and code style.
* Introduce a new test for tracking memory protection.
Sucessfully demonstrates that the bug causing write protection to be cleared by a read action has been fixed. (these tests fail on master)
* Address Comments
* Add host sync for SetReference
This ensures that any indirect draws will correctly flush any related buffer data written before them. Fixes some flashing and misplaced world geometry in MH rise.
* Make PageAlign static
* Re-enable read tracking, for reads.
* Support for resources on non-contiguous GPU memory regions
* Implement MultiRange physical addresses, only used with a single range for now
* Actually use non-contiguous ranges
* GetPhysicalRegions fixes
* Documentation and remove Address property from TextureInfo
* Finish implementing GetWritableRegion
* Fix typo
* Implement shader CC mode for ISCADD, X mode for ISETP and fix STS/STG with RZ
* Fix STG too and bump shader cache version
* Fix wrong name
* Fix Carry being inverted on comparison
* Interrupt GPU command processing when a frame's fence is reached.
* Accumulate times rather than %s
* Accurate timer for vsync
Spin wait for the last .667ms of a frame. Avoids issues caused by signalling 16ms vsync. (periodic stutters in smo)
* Use event wait for better timing.
* Fix lazy wait
Windows doesn't seem to want to do 1ms consistently, so force a spin if we're less than 2ms.
* A bit more efficiency on frame waits.
Should now wait the remainder 0.6667 instead of 1.6667 sometimes (odd waits above 1ms are reliable, unlike 1ms waits)
* Better swap interval 0 solution
737 fps without breaking a sweat. Downside: Vsync can no longer be disabled on games that use the event heavily (link's awakening - which is ok since it breaks anyways)
* Fix comment.
* Address Comments.
* Implement TreeMap from scratch.
Begin implementation of MemoryBlockManager
* Implement GetFreePosition using MemoryBlocks
* Implementation of Memory Management using a Tree.
Still some issues to work around, but promising thus far.
* Resolved invalid mapping issue.
Performance appears promising.
* Add tick metrics
* Use the logger instead
* Use debug loggin instead of info.
* Remove unnecessary code. Add descriptions of added functions.
* Improve memory allocation even further. As well as improve speed of position fetching.
* Add TreeDictionary to Ryujinx Commons
Removed Unnecessary Usigns
* Add a Performance Profiler + Improve ReserveFixed
* Begin transition to allocation in nvdrv
* Create singleton nvmemallocator
* Moved Allocation into Nv Related Files
As requested by gdkchan, any allocation of memory has been moved into the driver files.
Mapping remains in the GPU MemoryManager.
* Remove unnecessary usings
* Add missing descriptions
* Correct descriptions
* Fix formatting.
* Remove unnecessary whitespace
* Formatting / Convention Updates
* Changes / Fixes
Made syntax and convention changes as requested by gdkchan.
Fixed an issue where IsRegionUsed would return the wrong boolean.
Fixed an issue where GetFreePosition was asked for an address instead of a size.
* Undo commenting of Assert in shader cache
* Update Ryujinx.Common/Collections/TreeDictionary.cs
Co-authored-by: gdkchan <gab.dark.100@gmail.com>
* Resolved many suggestions
* Implement Improved TreeDictionary
Based off of Pseudo code and custom implementations.
* Rename _set to _dictionary
* Remove unused code
* Remove unused code.
* Remove unnecessary MapLow function.
* Resolve data-structure based issues
* Make adjustments to memory management.
Deactive de-allocation for now, it causes more harm than good.
* Minor refactorings + Re-implement deallocation
Also cleaned up unnecessary code.
* Add Tests for TreeDictionary
* Update data structure to properly balance the tree
* Experimental Implementation:
1. Reduce Time to Next Node to O(1) Runtime
2. Reduce While Loop Ct To 2 (In Most Cases)
* Address issues w/ Deallocating Memory
* Final Build
+ Fully Implement Dictionary Interface for new Data Structure
+ Cover All Memory Allocation Edge Cases, particularly w/ Games that De-Allocate a lot.
* Minor Corrections
Give TreeDictionary its own count (do not depend on inner dictionary)
Properly remove adjacent allocations
* Add AsList
* Fix bug where internal dictionary wasn't being updated w/ new node for overwritten key.
* Address comments in review.
* Fix issue where block wouldn't break out (Fixes UE4 issues)
* Update descriptions
* Update descriptions
* Reduce Node visibility to protect TreeDictionary Integrity + Remove usage of struct.
* Update tests to use new TreeDictionary implementation.
* Remove usage of dictionary in TreeDictionary
* Refactoring / Renaming
* Remove unneeded memoryblock class.
* Add space for while
* Add space for if
* Formatting / descriptions
* Clarified some descriptions
* Reduce visibility of memory allocator
* Edit method names to make more sense as memory blocks are no longer in use.
* Make names consistent.
* Protect against npe when sucessorof is called against keys that don't exist. (Not in use by memory manager, this is for other prs that might use this data structure)
* Possible edge-case resolve
* Update Ryujinx.Common/Collections/TreeDictionary.cs
Co-authored-by: gdkchan <gab.dark.100@gmail.com>
* Update Ryujinx.HLE/HOS/Services/Nv/NvMemoryAllocator.cs
Co-authored-by: gdkchan <gab.dark.100@gmail.com>
* Reduce # of unnecessary duplicate variables / Reduce visibility of variables only internally used.
* Rename count to _count
* Update Description of Add method.
* Fix copypasta
* Address comments
* Address comments
* Remove whitespace
* Address comments, condense variables.
* Consolidate vars
* Fix whitespace.
* Nit
* Fix exception msg
* Fix arrayIndex check
* Fix arrayIndex check + indexer
* Remove whitespace from cast
Co-authored-by: gdkchan <gab.dark.100@gmail.com>
* IPC refactor part 2: Use ReplyAndReceive on HLE services and remove special handling from kernel
* Fix for applet transfer memory + some nits
* Keep handles if possible to avoid server handle table exhaustion
* Fix IPC ZeroFill bug
* am: Correctly implement CreateManagedDisplayLayer and implement CreateManagedDisplaySeparableLayer
CreateManagedDisplaySeparableLayer is requires since 10.x+ when appletResourceUserId != 0
* Make it exit properly
* Make ServiceNotImplementedException show the full message again
* Allow yielding execution to avoid starving other threads
* Only wait if active
* Merge IVirtualMemoryManager and IAddressSpaceManager
* Fix Ro loading data from the wrong process
Co-authored-by: Thog <me@thog.eu>
This adds the guest GPU accessor to hashes computation.
As this change all the hashes from the cache, I added some migration
logic.
This is required for #1755.
* Allow copy destination to have a different scale from source
Will result in more scaled copy destinations, but allows scaling in some games that copy textures to the output framebuffer.
* Support copying multiple levels/layers
Uses glFramebufferTextureLayer to copy multiple layers, copies levels individually (and scales the regions).
Remove CopyArrayScaled, since the backend copy handles it now.
* shader cache: Fix Linux boot issues
This rollback the init logic back to previous state, and replicate the
way PTC handle initialization.
* shader cache: set default state of ready for translation event to false
* Fix cpu unit tests
* shader cache: Fix possible race causing crashes on manifest at startup
This fix a misplace function call ending up causing possibly two write
on the cache.info at the same time.
* shader cache: Make RemoveManifestEntries async too to be sure all operations are perform before starting the game
* shader cache: Fix invalid virtual address clean up
This fix an issue causing the virtual address of texture descriptors to
not be cleaned up when caching and instead cleaning texture format and swizzle.
This should fix duplicate high duplication in the cache for certain
games and possible texture corruption issues.
**THIS WILL INVALIDATE ALL SHADER CACHE LEVELS CONSIDERING THE NATURE OF THE ISSUE**
* shader cache: Address gdk's comment
* infra: Migrate to .NET 5
This migrate projects and CI to .NET 5
* Remove language version restrictions (now on 9.0 by default)
* infra: pin .NET 5 to avoid later issues
* infra: Cleanup csproj files
* infra: update dependencies
* infra: Add temporary workaround for a bug in Vector128.Create
see https://github.com/dotnet/runtime/issues/44704 for more informations
"Screen scissor" is the minimum size of all render targets, and is set when any render target is bound on NVN or OpenGL. Since it works on all active texture's real sizes, it is therefore more reliable than viewport 0's width, and is actually set before clear.
This fixes a regression with Hyrule Warriors: Age Of Calamity's cubemaps, which did not set viewport dimensions before clear. This resulted in attempting to create a cubemap with rectangular sides, which is logically and physically impossible. (also it just fails)
Here come Salieri, my implementation of a disk shader cache!
"I'm sure you know why I named it that."
"It doesn't really mean anything."
This implementation collects shaders at runtime and cache them to be later compiled when starting a game.
* Size hints for copy regions and viewport dimensions to avoid data loss
* Reword comment.
* Use info for the rule rather than calculating aligned size.
* Reorder min/max, remove spaces
* Only report that commands are available when the queue is not empty.
* Address Feedback
Co-authored-by: FICTURE7 <FICTURE7@gmail.com>
Co-authored-by: FICTURE7 <FICTURE7@gmail.com>
* Fix transform feedback errors caused by host pause/resume
* Fix TFB being used as something else issue with copies
* This is supposed to be StreamCopy
* WIP Range Tracking
- Texture invalidation seems to have large problems
- Buffer/Pool invalidation may have problems
- Mirror memory tracking puts an additional `add` in compiled code, we likely just want to make HLE access slower if this is the final solution.
- Native project is in the messiest possible location.
- [HACK] JIT memory access always uses native "fast" path
- [HACK] Trying some things with texture invalidation and views.
It works :)
Still a few hacks, messy things, slow things
More work in progress stuff (also move to memory project)
Quite a bit faster now.
- Unmapping GPU VA and CPU VA will now correctly update write tracking regions, and invalidate textures for the former.
- The Virtual range list is now non-overlapping like the physical one.
- Fixed some bugs where regions could leak.
- Introduced a weird bug that I still need to track down (consistent invalid buffer in MK8 ribbon road)
Move some stuff.
I think we'll eventually just put the dll and so for this in a nuget package.
Fix rebase.
[WIP] MultiRegionHandle variable size ranges
- Avoid reprotecting regions that change often (needs some tweaking)
- There's still a bug in buffers, somehow.
- Might want different api for minimum granularity
Fix rebase issue
Commit everything needed for software only tracking.
Remove native components.
Remove more native stuff.
Cleanup
Use a separate window for the background context, update opentk. (fixes linux)
Some experimental changes
Should get things working up to scratch - still need to try some things with flush/modification and res scale.
Include address with the region action.
Initial work to make range tracking work
Still a ton of bugs
Fix some issues with the new stuff.
* Fix texture flush instability
There's still some weird behaviour, but it's much improved without this. (textures with cpu modified data were flushing over it)
* Find the destination texture for Buffer->Texture full copy
Greatly improves performance for nvdec videos (with range tracking)
* Further improve texture tracking
* Disable Memory Tracking for view parents
This is a temporary approach to better match behaviour on master (where invalidations would be soaked up by views, rather than trigger twice)
The assumption is that when views are created to a texture, they will cover all of its data anyways. Of course, this can easily be improved in future.
* Introduce some tracking tests.
WIP
* Complete base tests.
* Add more tests for multiregion, fix existing test.
* Cleanup Part 1
* Remove unnecessary code from memory tracking
* Fix some inconsistencies with 3D texture rule.
* Add dispose tests.
* Use a background thread for the background context.
Rather than setting and unsetting a context as current, doing the work on a dedicated thread with signals seems to be a bit faster.
Also nerf the multithreading test a bit.
* Copy to texture with matching alignment
This extends the copy to work for some videos with unusual size, such as tutorial videos in SMO. It will only occur if the destination texture already exists at XCount size.
* Track reads for buffer copies. Synchronize new buffers before copying overlaps.
* Remove old texture flushing mechanisms.
Range tracking all the way, baby.
* Wake the background thread when disposing.
Avoids a deadlock when games are closed.
* Address Feedback 1
* Separate TextureCopy instance for background thread
Also `BackgroundContextWorker.InBackground` for a more sensible idenfifier for if we're in a background thread.
* Add missing XML docs.
* Address Feedback
* Maybe I should start drinking coffee.
* Some more feedback.
* Remove flush warning, Refocus window after making background context
* Use a better viewport flipping approach
* New approach to detect depth mode
* nit: Sort method on the OpenGL backend
* Adjust spacing on comment
* Unswap near and far parameters based on ScaleZ
* Add swizzle matching rules.
Improves rules which try to match incompatible formats as perfect, such as D32 float -> R32 float.
Remove Format.HasOneComponent, since this information is now available via the FormatInfo struct.
* Fix this rule.
* Update component counts for depth formats.
* Initial implementation. Still pending better valid-overlap handling,
disposed pool, compressed format flush fix.
* Very messy backend resource cache.
* Oops
* Dispose -> Release
* Improve Release/Dispose.
* More rule refinement.
* View compatibility levels as an enum - you can always know if a view is only copy compatible.
* General cleanup.
Use locking on the resource cache, as it is likely to be used by other threads in future.
* Rename resource cache to resource pool.
* Address some of the smaller nits.
* Fix regression with MK8 lens flare
Texture flushes done the old way should trigger memory tracking.
* Use TextureCreateInfo as a key.
It now implements IEquatable and generates a hashcode based on width/height.
* Fix size change for compressed+non-compressed view combos.
Before, this could set either the compressed or non compressed texture with a size with the wrong size, depending on which texture had its size changed. This caused exceptions when flushing the texture.
Now it correctly takes the block size into account, assuming that these textures are only related because a pixel in the non-compressed texture represents a block in the compressed one.
* Implement JD's suggestion for HashCode Combine
Co-authored-by: jduncanator <1518948+jduncanator@users.noreply.github.com>
* Address feedback
* Address feedback.
Co-authored-by: jduncanator <1518948+jduncanator@users.noreply.github.com>
* Silence several build warnings
* Remove fixed buffers from NVDEC struct
* Remove unused field and usings
* Fix wrong name
* Silence more warning on H264 PictureInfo
* Logger class changes only
Now compile-time checking is possible with the help of Nullable Value
types.
* Misc formatting
* Manual optimizations
PrintGuestLog
PrintGuestStackTrace
Surfaceflinger DequeueBuffer
* Reduce SendVibrationXX log level to Debug
* Add Notice log level
This level is always enabled and used to print system info, etc...
Also, rewrite LogColor to switch expression as colors are static
* Unify unhandled exception event handlers
* Print enabled LogLevels during init
* Re-add App Exit disposes in proper order
nit: switch case spacing
* Revert PrintGuestStackTrace to Info logs due to #1407
PrintGuestStackTrace is now called in some critical error handlers
so revert to old behavior as KThread isn't part of Guest.
* Batch replace Logger statements
* Add new structures from official docs, start migrating GPFifo
* Finish migration to new GPFifo processor
* Implement fast constant buffer data upload
* Migrate to new GPFifo class
* XML docs
* Initial transform feedback support
* Some nits and fixes
* Update ReportCounterType and Write method
* Can't change shader or TFB bindings while TFB is active
* Fix geometry shader input names with new naming
* Initial NVDEC and VIC implementation
* Update FFmpeg.AutoGen to 4.3.0
* Add nvdec dependencies for Windows
* Unify some VP9 structures
* Rename VP9 structure fields
* Improvements to Video API
* XML docs for Common.Memory
* Remove now unused or redundant overloads from MemoryAccessor
* NVDEC UV surface read/write scalar paths
* Add FIXME comments about hacky things/stuff that will need to be fixed in the future
* Cleaned up VP9 memory allocation
* Remove some debug logs
* Rename some VP9 structs
* Remove unused struct
* No need to compile Ryujinx.Graphics.Host1x with unsafe anymore
* Name AsyncWorkQueue threads to make debugging easier
* Make Vp9PictureInfo a ref struct
* LayoutConverter no longer needs the depth argument (broken by rebase)
* Pooling of VP9 buffers, plus fix a memory leak on VP9
* Really wish VS could rename projects properly...
* Address feedback
* Remove using
* Catch OperationCanceledException
* Add licensing informations
* Add THIRDPARTY.md to release too
Co-authored-by: Thog <me@thog.eu>
