Ticket Information - ID: #1313
| ID: | Category: | Severity | Reproducibility | Date Submitted | Updated By: |
|---|---|---|---|---|---|
| 0001313 | Bug Reporting | normal | sometimes | 02/19/24 03:14PM | Hawker |
|
|
| Summary: | Favorites API endpoint inconsistently sorting results |
| Description: | On a multiple occasions the Favorites API will return results either sorted by the wrong column (favorite/post-id vs added) or sorted in the wrong direction (ascending vs descending). The Favorites API should be consistent in returning results sorted by date added, and whether ascending or descending order (which way is irrelevant, as long as it stays consistent). Previous issues: /index.php?page=forum&s=view&id=6614 /index.php?page=forum&s=view&id=6788 (current issue) When this started I was getting results sorted by favorite/post-id (which isn't useful), but as of this writing, I'm getting results sorted by date added ("added") descending - which makes a lot of sense, as it puts new additions at the front of the results. This is easy enough for me to script, but breaks gallery-dl (which is assuming ascending order based on past API behavior). I use gallery-dl and my own scripts to keep my favorites cached locally (gallery-dl for better metadata, my scripts for custom organization). |
| Additional Info: | Gallery-dl specific information: The most consistent behavior for a long time was "sort by time added, ascending" (this is what gallery-dl uses). You can see gallery-dl assume this in its verbose output (cleaned up a bit for posting here): [gelbooru][debug] Using GelbooruFavoriteExtractor for '/index.php?page=favorites&s=view&id=17627' [urllib3.connectionpool][debug] Starting new HTTPS connection (1): gelbooru.com:443 [urllib3.connectionpool][debug] https://gelbooru.cc:443 "GET /index.php?page=dapi&q=index&json=1&s=favorite&id=17627&limit=1&api_key=[redacted]&user_id=17627 HTTP/1.1" 200 None [urllib3.connectionpool][debug] https://gelbooru.cc:443 "GET /index.php?page=dapi&q=index&json=1&s=favorite&id=17627&limit=100&api_key=[redacted]&user_id=17627&pid=236 HTTP/1.1" 200 None [urllib3.connectionpool][debug] https://gelbooru.cc:443 "GET /index.php?page=dapi&q=index&json=1&id=3850872&s=post&api_key=[redacted]&user_id=17627 HTTP/1.1" 200 None [urllib3.connectionpool][debug] https://gelbooru.cc:443 "GET /index.php?page=post&s=view&id=3850872 HTTP/1.1" 200 None [urllib3.connectionpool][debug] https://gelbooru.cc:443 "GET /index.php?page=dapi&q=index&json=1&id=3851028&s=post&api_key=[redacted]&user_id=17627 HTTP/1.1" 200 None [urllib3.connectionpool][debug] https://gelbooru.cc:443 "GET /index.php?page=post&s=view&id=3851028 HTTP/1.1" 200 None [urllib3.connectionpool][debug] https://gelbooru.cc:443 "GET /index.php?page=dapi&q=index&json=1&id=3852413&s=post&api_key=[redacted]&user_id=17627 HTTP/1.1" 200 None [urllib3.connectionpool][debug] https://gelbooru.cc:443 "GET /index.php?page=post&s=view&id=3852413 HTTP/1.1" 200 None [urllib3.connectionpool][debug] https://gelbooru.cc:443 "GET /index.php?page=dapi&q=index&json=1&id=3852432&s=post&api_key=[redacted]&user_id=17627 HTTP/1.1" 200 None [urllib3.connectionpool][debug] https://gelbooru.cc:443 "GET /index.php?page=post&s=view&id=3852432 HTTP/1.1" 200 None [urllib3.connectionpool][debug] https://gelbooru.cc:443 "GET /index.php?page=dapi&q=index&json=1&id=3852730&s=post&api_key=[redacted]&user_id=17627 HTTP/1.1" 200 None [urllib3.connectionpool][debug] https://gelbooru.cc:443 "GET /index.php?page=post&s=view&id=3852730 HTTP/1.1" 200 None [urllib3.connectionpool][debug] https://gelbooru.cc:443 "GET /index.php?page=dapi&q=index&json=1&id=3855734&s=post&api_key=[redacted]&user_id=17627 HTTP/1.1" 200 None [urllib3.connectionpool][debug] https://gelbooru.cc:443 "GET /index.php?page=post&s=view&id=3855734 HTTP/1.1" 200 None It gets the count of favorites (the initial "limit=1" call), then uses that to determine the offset to the end of the results, and starts checking for new posts to download. I have it configured to compare against a download archive, and skip any that are already present. My understanding is it stores image URLs in the archive, so it has to retrieve the post first to get the image URL and compare against the archive. (I'll submit a bug that it really shouldn't check for metadata until after checking the image against the archive; that should get rid of the extra json calls.) Troubleshooting this with the author of gallery-dl has been difficult, as we've observed the API returning results differently based on the user-id being downloaded (https://github.com/mikf/gallery-dl/discussions/4769 - his user-id returned as expected while mine returned sorted in a different order) and there does not seem to be any official stance on what order the API should be returning - just past custom and observation. |
Hawker replied at 2024-02-19 16:03:37  |
| I'm also following some of these issues - especially the needless API calls on downloaded posts - in a gallery-dl issue here: https://github.com/mikf/gallery-dl/issues/5220 |
Hawker replied at 2024-03-24 15:05:06 |
| And it seems this is working as expected again - favorites API is returning them in descending order of date-added, so the most recently added favorites are returned first. This is excellent; I'd just like to see it be consistent and documented. |
