4a929bd8e2
Optimize scanning using directory name hint
...
As discussed in issue #5440 , directory scans are slow when compressed files must be checked against multiple databases. This commit adds support to hint to the scanner that a particular database should be used so that only a single scan is required. If the directory name being scanned matches the database name then that single database is selected.
2017-09-19 16:58:28 -07:00
6ac4486f4d
Fix typo from 9a76a93ea that broke the build
2017-09-19 16:49:45 -07:00
9a76a93ea8
OSX buildfixes/cleanups
2017-09-19 21:37:51 +02:00
e8f39f004e
MSVC buildfixes
2017-09-19 21:29:28 +02:00
ee06a8f175
Merge pull request #5447 from bkoropoff/scanning2
...
More scanning rework
2017-09-19 15:46:04 +02:00
fb8fe84684
Fix archive scanning
...
Archive scans were broken for cores like Atari 5200 and Commodore 64 because archive contents were skipped if a core reported it supported compressed files. The problem is, it's a false assumption that just because a core supports compressed files that the content database contains CRCs for compressed files.
Instead of adding the contents of every compressed file to the scan list ahead of time, the patched logic now checks for the CRC of the archive itself as it always has but then adds the archive contents to the scan list if the archive itself is no match. This maintains the logic of adding only the archive if the core supports compressed files and the archive CRC matches but also allows for deeper content scans when there is no match.
The patch also removes `core_info_unsupported_content_path` as it was redundant with existing `core_info_database_supports_content_path` logic.
2017-09-18 23:15:28 -07:00
d4beeeda33
task_database: skip serials for the right file type!
2017-09-18 09:03:43 -07:00
84edc8ffb0
task_database: prune files referenced from cue/gdi files
...
Since we use cue/gdi files to find the right track file to
scan, don't bother scanning them separately.
2017-09-18 09:03:43 -07:00
6c1044a699
task_database: support GDI files
2017-09-18 09:03:43 -07:00
fa6d0fac67
task_database: more scanning rework
...
- Use the primary (largest data) track for CRC identification, as it
seems least likely to be shared among multiple discs
- CRC the primary track when importing a loose .cue file.
- If multiple tracks are in the same file, CRC the correct chunk
2017-09-18 09:03:43 -07:00
49cf80db10
Fix resource leaks
2017-09-18 15:57:05 +02:00
6648ebed0d
Silence some warnings
2017-09-18 11:56:59 +02:00
5a3969d4fa
task_database: rework cd scanning
...
Support chd files. Fall back to using the CRC32 of the last track
of CDs if a serial number is not available. This requires new
database files, but is necessary for chd scanning to work.
2017-09-17 02:11:07 -07:00
b51a19da5c
Fix memory leak
2017-09-12 05:26:59 +02:00
c1c42cac6b
(task_database.c) Reduce stack usage
2017-09-12 05:00:22 +02:00
48550235d5
Fix slow content scans.
...
The check for unsupported content was reversed which was causing files to be checked against every database. Correcting this improves performance drastically. This appears to have been broken since at least commit #fc1f2cdcf.
2017-09-06 20:51:37 -07:00
50b838d8ed
Cleanups
2017-07-30 10:11:41 +02:00
bbb2a7d6f4
Add serial detection for Wii
2017-07-29 23:23:05 -04:00
5733d8ebd5
move content list builder into scanner task with progress, fixes menu freeze with large playlists
2017-07-09 00:44:02 -04:00
634de19dc9
(WiiU) enable libretrodb.
2017-05-22 06:07:27 +01:00
eede1a4978
no need to swap endianess here.
2017-05-22 05:26:13 +01:00
8eca08c6ac
Start using string_is_equal_fast/string_is_not_equal_fast macros
...
instead of straight memcmp
2017-05-16 03:20:59 +02:00
80d9d1f143
Rewrite task_queue
2017-05-14 20:43:48 +02:00
72bd83b8d1
Cleanups to tasks_internal.h
2017-05-13 18:08:37 +02:00
42ebe7a7ad
Merge runloop.c and retroarch.c
2017-05-11 09:11:46 +02:00
f9bc2d3783
Use memcmp instead of string_is_equal where possible for
...
better performance
2017-04-21 22:47:15 +02:00
ff575a3460
PCE-CD magic number detection
2017-03-07 01:16:58 +01:00
10bf8d4a8a
Backport enlarge char buffers
2017-02-27 09:28:38 +01:00
96c8ca5a09
Header update #1
2017-01-22 13:40:32 +01:00
a3f5590f41
remove dependence on configuration.h inside task_database.c
2017-01-18 01:23:24 +01:00
2834370e09
Revert "allow serial scanner to match PBP"
2017-01-07 05:52:12 +01:00
1866706c75
allow serial scanner to match PBP
2017-01-06 22:05:55 -05:00
6faabfeebf
Don't use settings struct from thrading functions
2016-12-31 07:03:29 +01:00
41f40acfc4
guard task struct read/writes across threads with a mutex, fixes several data races found by ThreadSanitizer and helgrind
2016-12-29 00:54:12 -05:00
958a028b0a
(task_database.c) Remove unused message queue
2016-12-24 20:36:37 +01:00
77e5cdbfde
Revert "Rename runloop_msg_queue_push to video_driver_msg_queue_push"
...
This reverts commit ad7386913c
2016-12-22 23:36:11 +01:00
ad7386913c
Rename runloop_msg_queue_push to video_driver_msg_queue_push
...
and move code to video_driver.c
2016-12-22 13:15:02 +01:00
c8dc0fef0c
Cleanups - turn some functions into static functions and move
...
to task_database.c
2016-12-20 21:08:31 +01:00
d0cc1afe0a
(tasks) Style nits
2016-12-04 19:12:35 +01:00
6cc14a4fb9
Add missing copyright mention on some files
2016-10-27 18:17:43 +02:00
d1d5584d18
update copyright
2016-10-27 11:47:10 -04:00
8e329a2072
Merge pull request #3781 from frangarcj/master
...
(VITA) Working scanning content
2016-10-13 17:51:13 +01:00
4c19543427
(VITA) Workaround for scanning content
2016-10-13 18:00:40 +02:00
a4b90bcede
Set msg[127] = 0
2016-10-13 13:33:51 +02:00
88c8c49ef8
skip scanning of compressed files within archives if the core attached to this database supports archive files themselves
2016-10-12 16:14:00 -04:00
481ebced22
Prevent implicit memsets
2016-10-09 09:56:03 +02:00
1388220866
(tasks) Don't do implicit memsets in crucial performance-oriented code
2016-10-08 18:31:43 +02:00
fc1f2cdcf4
only scan the content databases that support the file's extension
2016-10-03 16:09:10 -04:00
c8dc941c42
iso_get_serial - cleanup
2016-09-29 11:10:52 +02:00
a304b42f73
(task_database.c) Cleanup
2016-09-29 11:07:56 +02:00
1842e35b8d
Cleanups
2016-09-29 11:03:59 +02:00
cfe91125db
Create database_info_get_current_element_name
2016-09-29 10:59:50 +02:00
b286703a81
Cleanup task_database_handler
2016-09-29 10:55:50 +02:00
7e1639f33b
Create database_info_set_type
2016-09-29 10:43:38 +02:00
fc3c17547e
Use database_info_get_type
2016-09-29 10:41:30 +02:00
24f34431ee
Header include cleanups
2016-09-29 10:37:31 +02:00
94b0dd36ab
remove unused variable
2016-09-25 15:51:46 +02:00
af98ee1c8a
Add CRC calculation function that works with archives with or without a path inside (first file is used if no path)
...
Add all archive's contents to database scan list when scanning files and directories
Allow scanning a single file that is an archive
Remove unnecessary prototypes from archive_file.h
Simplify retrieving of CRCs from archives when scanning
2016-09-25 00:15:05 -04:00
3f35e10014
Solve some warnings pointed out by scan-build
2016-09-25 05:55:55 +02:00
55517dbc12
Prevent warning
2016-09-21 02:17:32 +02:00
b7735520f2
fix scanning of archive files
2016-09-20 15:40:29 -04:00
6ac73c5655
use generic crc32 to scan files with
2016-09-20 13:34:56 -04:00
992b87321a
more archive paths
2016-09-19 21:06:23 -04:00
137968c001
treat .apk files as .zip, set archive_path any time we need to decompress a file
2016-09-19 18:38:33 -04:00
6544b0bcd7
Get rid of HAVE_LIBRETRODB ifdefs
2016-09-18 21:50:39 +02:00
a7ffead8f6
fix more memory corruption
2016-09-18 15:20:27 -04:00
dc4760f5a6
fix another wrong type
2016-09-18 14:39:37 -04:00
f13bce4e50
- use the detected stream backend's crc32 function
...
- remove zlib/7zip ifdef from archive_file.c, task_content.c and task_database.c
- don't re-compute CRC from one stored in archive
2016-09-18 12:31:24 -04:00
7c29fd6c33
create 7z archive backend, enables scanning of 7z content
2016-09-18 10:31:21 -04:00
acd4dd527b
Header include cleanups
2016-09-05 18:31:32 +02:00
397f82c9f5
Get rid of configuration.h dependency in list_special.c
2016-07-18 07:15:58 +02:00
21134eb28c
Create FILE_PATH_DETECT
2016-06-30 06:07:44 +02:00
fbc83640b6
More translatable strings
2016-06-30 05:58:05 +02:00
6756501dac
Create FILE_PATH_LUTRO_PLAYLIST
2016-06-30 05:19:25 +02:00
0f18ade02a
Add .lutro scanning
...
When a file wih the .lutro extension is found, the ROM is added to Lutro.lpl. The name of the game is the name of the file without the extension.
2016-06-29 15:07:12 +02:00
ca3eecf5c0
Create fill_pathname_join_noext
2016-06-28 13:05:46 +02:00
2dddc3fc88
Add FILE_PATH_LPL_EXTENSION
2016-06-26 10:12:28 +02:00
dee7533d65
(libretro-common) Start creating STRING_REP_ULONG
2016-06-23 07:37:02 +02:00
c3eda56561
Move hashes over to msg_hash.c
2016-06-20 17:55:34 +02:00
68bf755fe7
Remove two unused hashes
2016-06-20 16:19:30 +02:00
2ebd6e6f49
Header cleanups
2016-06-20 06:34:30 +02:00
34ad3bd103
Cleanups
2016-06-03 05:24:21 +02:00
65d5ebc2bb
Fix some dereference after null check warnings
2016-06-02 20:40:51 +02:00
da6ac4aae7
Fix explicit NULL dereferenced warnings
2016-06-01 04:05:14 +02:00
7d26034ebe
Rename rarch_task_* to task_*
2016-05-27 18:14:47 +02:00
a98616b45b
Fix explicit NULL dereferenced/dereference before null check warnings
2016-05-26 17:47:21 +02:00
57f797dad6
Cleanup
2016-05-21 13:16:48 +02:00
5555d18a52
Cleanup
2016-05-17 14:24:43 +02:00
449c7483cd
Rename content_playlist_ to playlist_
2016-05-16 09:07:44 +02:00
7c1938b674
Rename task functions
2016-05-13 10:19:53 +02:00
7dffc6286b
Internally change directory settings
2016-04-28 19:26:02 +02:00
86cf3243aa
Fix arcade LPL generation
2016-04-15 05:55:11 +07:00
4f74976cac
Arcade scanning
2016-04-15 05:28:44 +07:00
e41127459e
Rename 'filestream' functions
2016-03-24 04:09:25 +01:00
ac6362cae4
Combine list_special files
2016-03-20 17:28:24 +01:00
93b7dc1bdd
(libretro-common) move 'lists' files to lists/
2016-03-20 14:53:54 +01:00
4b9143430c
Add C89 workaround
2016-02-25 22:26:48 +01:00
f8b866c8c1
Combine file_ops.c and content.c
2016-02-15 21:49:29 +01:00
5b6d310abd
Rename enums
2016-02-09 18:26:27 +01:00
6e9bc55dd6
Rename task_ctl to task_queue_ctl
2016-02-09 17:51:51 +01:00
08eb091610
Rename rarch_ to retro_
2016-02-09 17:47:04 +01:00
ce2869b4d8
Create tasks_internal and split up RA-specific task code to outside
2016-02-09 17:12:39 +01:00
0b1b56214e
Try to reduce dependence on general.h
2016-02-07 13:18:48 +01:00
f1ca9604b7
Header include changes
2016-02-07 01:50:08 +01:00
0643b83f7d
Header include changes
2016-02-05 13:57:08 +01:00
2251517774
Respect 80-char width limit
2016-02-04 10:25:22 +01:00
16ba94b531
Revert "(task_database.c) Style nits"
...
This reverts commit 0f46d42c26
2016-02-04 10:17:54 +01:00
0f46d42c26
(task_database.c) Style nits
2016-02-03 17:31:19 +01:00
9ab31824f8
Use retro_read_file in more places
2016-01-30 03:58:33 +01:00
b47caeb58b
Refactor away runloop_msg_queue_push_new
2016-01-29 14:30:09 +01:00
b807b89837
Add TASK_CTL_PUSH
2016-01-28 09:42:55 +01:00
40e2994fb6
remove log message
2016-01-26 15:24:17 -05:00
2af0708047
do not push entries that are already on the database
2016-01-26 15:22:16 -05:00
bb36ad2772
Take out playlist.h header include from database_info.h
2016-01-25 07:06:58 +01:00
9459cf6445
Rename zlib_ to file_archive_
2016-01-24 22:17:11 +01:00
aae1faf76f
Move crc32_calculate to file_archive_zlib.c
2016-01-24 21:50:28 +01:00
3c2a45d923
Rename file_extract functions
2016-01-24 07:42:46 +01:00
c6f7afbd9a
Adjust zlib_crc32_calculate
2016-01-24 06:47:06 +01:00
ae2f6f06a0
rarch_task_push_dbscan - prevent possible memleaks
2016-01-21 03:25:00 +01:00
454abdee48
stdstring - create string_is_equal
2016-01-20 04:07:24 +01:00
7a55ae3765
Start updating headers for 2016
2016-01-10 04:06:50 +01:00
1439a36174
(tasks) Start using string_is_empty
2015-12-26 07:23:13 +01:00
2d55dc3bda
Complete namespace changes
2015-12-07 15:32:14 +01:00
9471020bf7
Reduce some more msg char variables
2015-12-06 15:55:05 +01:00
058e941d8a
Move header include out of tasks.h
2015-12-04 13:19:53 +01:00
29e9b87319
(tasks) Make task handles honor task->cancelled
2015-11-24 22:18:09 -03:00
c342bb437a
Fix warnings
2015-11-23 23:47:25 +01:00
c21ffeff71
Reimplement content scanning using tasks
2015-11-23 17:42:01 -03:00
074f14b132
Add verbosity.c/verbosity.h
2015-11-23 12:03:38 +01:00
1cd1ecfc5c
More C89_BUILD buildfixes
2015-11-19 13:43:48 +01:00
0e8d65c44e
(task_database) Fix serial lookup
2015-11-16 20:49:51 -03:00
0ae631078a
(task_database) Stop crc lookup on first match
2015-11-14 19:44:08 -03:00
9e2dc0745e
Add param to dir_list_new_special
2015-10-26 21:30:58 +01:00
02a43c6b2a
Change 'rarch' to 'retro' namespace in libretro-common
2015-10-26 03:18:13 +01:00
02a5d738fe
Static code analysis cleanups
2015-09-28 23:00:22 +02:00
1eb80f6c93
More static code analysis fixes
2015-09-28 17:18:48 +02:00
f44852b35b
Fix redefinition of variable
2015-09-28 03:32:05 +02:00
f2f3374760
Get rid of unused variable
2015-09-28 03:31:09 +02:00
c7fe8922a1
Allow upper-case CUE extension too
2015-09-27 03:50:45 +02:00
1d33bf5db7
Add uppercase 'ISO' extension too
2015-09-27 03:48:58 +02:00
d2d402ea1b
Refactor CD scanning, and detect PSP games before searching serials
2015-09-27 02:14:24 +02:00
1ffe9eae43
Implement PSP scanning
2015-09-27 00:12:17 +02:00
938e89562b
Search the ps1 serial in the whole binary
2015-09-25 00:36:12 +02:00
fea95aa4a1
Finish implementing psx scanning
2015-09-24 14:14:42 +02:00
0e7018c27d
Serial lookup for psx scanning
2015-09-22 15:22:15 +02:00
432aac58a3
Get rid of more fcntl header includes
2015-09-18 04:03:11 +02:00
78185be8bf
Build fixes
2015-09-16 22:44:16 +02:00
fad78d8b81
Split up code to task_database_cue.c
2015-09-16 22:39:47 +02:00
07f362e88d
Start adding ps1 scanning
2015-09-16 22:11:08 +02:00
8ca8b43545
Take out retro_endianness.h include out of retro_miscellaneous.h
2015-09-14 03:34:05 +02:00
9554597160
Add missing header includes, fixes CXX_BUILD.
2015-09-06 14:46:25 +02:00
f88575633e
Cleanups
2015-09-05 20:34:22 +02:00
c05bb6e0c3
Tolerate unset database path.
2015-08-20 19:24:08 +02:00
28b1c9eada
Rename this variable, every other instance of it has the same name.
2015-08-20 19:08:07 +02:00
c4ff5e73f1
(task_database.c) Refactor away get_ptr function
2015-08-06 04:14:49 +02:00
f967c88ff0
(runloop_data.c) Cleanups
2015-08-06 02:31:50 +02:00
9185d37710
Bump up collection limit
2015-07-26 00:40:24 +02:00
51137d5351
Set pending_scan_finished to false in rarch_main_data_db_uninit
2015-07-09 02:33:46 +02:00
1c406bc098
Add rarch_main_data_pending_scan_finished and other mechanisms
...
so that the call to menu_environment_cb gets made on main thread
2015-07-09 02:31:46 +02:00
51e2a128c5
(runloop data) Don't have to pass around runloop pointer anymore
2015-07-08 23:01:11 +02:00
52e2fd1689
Refactor task_database.c
2015-07-08 22:03:23 +02:00
c84ddb53fa
(task_databse.c) Refactors
2015-07-08 21:48:58 +02:00
039ba91c48
Add mechanism so that task_database informs the menu driver that
...
we need to refresh the horizontal list. Kivutar; make the icons
work again after we do a horizontal list refresh
2015-07-08 01:01:41 +02:00
a3aab2f98e
(task_database) Dont call zlib_parse_file_iterate_stop so much
2015-07-05 10:40:51 -03:00
3cc6e07174
(task_database) Fix leaks when scanning zip files
2015-07-05 10:23:21 -03:00
d4dcf784c9
Update msg_hash.c
2015-07-01 19:40:50 +02:00
5d576ce61d
(task_database.c) Use fill_pathname_join_delim in task_database.c
2015-06-22 19:51:13 +02:00
5f67c53e6a
Use fill_short_pathname_representation
2015-06-22 19:39:56 +02:00
b08c399d3b
(task_database.c) Fix some warnings
2015-06-15 22:48:41 +02:00
a86977de3a
(tasks) Use hashes
2015-06-14 17:11:48 +02:00
25372dfb56
(task_database.c) Refactor to not use strcmp
2015-06-14 16:59:38 +02:00
a10cdfa8b1
(tasks) Cleanups
2015-06-12 22:37:06 +02:00
85800a81b6
(task_database.c) Use a query for faster CRC scanning
2015-06-10 14:51:26 -03:00
4d23fd211c
(libretro-db) Store CRC32 as uint32_t
2015-06-09 20:21:14 -03:00
b321c38d62
(task_database.c) If 0 out some excessive logging
2015-06-09 21:10:57 +02:00
9487def760
(task_database.c) We can now scan individual files
2015-06-09 19:38:51 +02:00
ba0e25dd88
Rename database_info_init to database_info_dir_init
2015-06-09 18:53:18 +02:00
29e3f5ff66
Expand playlist format - rename it to 'lpl'
2015-06-08 23:38:03 +02:00
4a37c4e020
Remove duplicate strcasestr
2015-06-02 16:49:35 +02:00
0a6be4c82e
(task_database.c) Fix database_info_iterate_start for Windows
2015-05-28 08:57:50 +02:00
fe5880355d
database_info_list_iterate_next - set db_state->info to NULL
...
after freeing
2015-05-28 02:55:39 +02:00
8bde46deb9
Free db_state->info first in database_info_list_iterate_new
2015-05-28 01:25:42 +02:00
0e23908b78
(Task database) Should fix ZIP playlist scanning
2015-05-27 07:23:09 +02:00
65b8b2e0cb
(Database dir scanning) Can now add ZIP entry to collection
2015-05-27 04:42:22 +02:00
d72df8492b
(Task database) Compare first entry of ZIP file
2015-05-27 03:23:27 +02:00
aa56b8e1e3
(Task database) Start using zlib_parse_file_iterate in task_database.c
2015-05-27 01:36:15 +02:00
d33ee0ca90
(task_database.c) Update TODO notes
2015-05-26 09:04:57 +02:00
c94e29ed02
Create rarch_strcasestr
2015-05-26 02:31:29 +02:00
d275f8dd8f
Database dir scanning now creates 'rpl' (retroarch playlist) files
2015-05-26 00:25:07 +02:00
8fb8887c34
Create new playlist format
2015-05-26 00:12:49 +02:00
e05c0ab4f4
Set 'display name' to crc
2015-05-25 21:39:50 +02:00
350a33a4ae
Start implementing database_info_list_iterate_found_match
2015-05-25 18:46:36 +02:00
1a85dfe60a
(Database) Remove some unused variables
2015-05-25 10:04:54 +02:00
0242b38c12
(task_database.c) Cleanups
2015-05-25 07:54:44 +02:00
23299eb74d
(task_database.c) Do string compare of CRC
2015-05-25 07:34:42 +02:00
0f58ad0e4e
(Task database) Experimental database scanning
2015-05-25 07:05:00 +02:00
76e1c2d965
database_info_iterate_crc_lookup - cleanup
2015-05-25 05:36:23 +02:00
32412ab918
Simplify rarch_main_data_db_cleanup_state
2015-05-25 03:46:03 +02:00
befa2f111c
Create database_info_iterate_crc_lookup
2015-05-25 03:20:43 +02:00
3c6dd65208
Pass db_state to zlib_parse_file
2015-05-24 08:06:29 +02:00
Aaron Oneal