PixivForMuzei3’s arbitrary bookmark display

Preamble

Recently, I had received two requests in as many weeks to improve how my app selects bookmarked artworks to be the wallpaper.
On a roll with app updates and under some ADHD induced hyper fixation, I set out to improve this feature which had been neglected for over a year.

Currently my app would only select artworks from the 30 or so most recently bookmarked ones, which is not optimal. Ideally the app would be able to choose from any bookmark ever made and present that masterpiece you loved two years ago as your wallpaper.

Investigation

I knew that you could already query the number of bookmark artworks a user account has by making a call to the https://app-api.pixiv.net/v1/user/detail endpoint and then see what value of total_illust_bookmarks_public was present.

A simple implementation formed in my mind: get the number of total bookmarks, select a random number beween 0 and the total, then query the API to get bookmark #1234, and Bob’s your uncle.

This line of thinking immediately ran into a stumbling block: the Pixiv API had no mechanism to simply give to you an arbitrary bookmark.
The Pixiv mobile application only allows you to browse your bookmarks by scrolling through an endlessly paginating list of artworks, and the API design reflects this.
Questions of poor user design aside, this meant I had to find a investigate a little harder to solve the user requests.

Deeper Investigation

A request to the bookmarks endpoint returns an array containing the metadata for 30 artworks and a URL to fetch the next set of 30 artworks. Of immediate interest is the max_bookmark_id parameter.

{
    "illusts":[
        {artwork0},
        {artwork1},
        ..
        ..
        {artwork29}
    ],
    "next_url": "https://app-api.pixiv.net/v1/user/bookmarks/illust?restrict=public&user_id=12345678&max_bookmark_id=13765616633"
}

Successive requests to the bookmark endpoint using the parameters suggested in next_url would indeed return a different set of 30 artworks that immediately picked up where the previous set of artworks left off, and a different max_bookmark_id

{
    "illusts":[
        {artwork30},
        {artwork31},
        ..
        ..
        {artwork59}
    ],
    "next_url": "https://app-api.pixiv.net/v1/user/bookmarks/illust?restrict=public&user_id=12345678&max_bookmark_id=13563631106"
}

Clearly, this parameter was how the pagination was being controlled. The next step was determining what the max_bookmark_id paramter represented and how I could use it.

Some things I noticed after some trial and error varying the parameter:

• max_bookmark_id would decrease as we made sucessive calls with the suggested max_bookmark_id parameter and looked at older artworks.
• max_bookmark_id‘s value is consistent across accounts, so it is a global variable serverside.
• max_bookmark_id is NOT the Unix epoch timestamp.
• Making a call with a larger max_bookmark_id than the original would just return the latest set of artworks
• Making a call with a 0 or otherwise very small max_bokmark_id would return an empty set and no next_url parameter
• I discovered by chance a value of max_bookmark_id that returned a valid set of artworks but no next_url value. This corresponded to the very first artworks I had bookmarked on this account.

max_bookmark_id appears to be a timestamp counting in centiseconds from an epoch in mid 2017, and controls a “sliding window” into your list of bookmarks.This obviously doesn’t seem right, but this was enough knowledge to work on and I was pretty done with investigating max_bookmark_id any further.

The last point was the ticket, once I found a special value of max_bookmark_id that corresponded to when the first artworks were bookmarked I could then arbitrarily pick any bookmarked artwork.
This value would differfrom user to user, so I need to find a generalised solution, and an efficient one so I don’t tick off Pixiv too much.

Implementation

So we have a finite search space that is filled with discrete elements and is already sorted for us, sounds like an excellent time to break out the binary search!

The search parameters spelled out:

• Start value: current max_bookmark_id
• Target value: An API response that contains a non-empty set of artworks while not possessing a next_url value.
• Heuristics:
  • If the API response includes a next_url parameter, then we are too high. Subtract step size to the current search point and try again.
  • If the API response includes no artworks, then we are too low. Add the step size to the current search point and try again.

The initial value is the current max_bookmark_id to handle the case where the user has less than the one page of bookmarks. In this case, the algorithm immediately exits and returns the

The oldest valid max_bookmark_id is then saved.

Simplified example of the real code, full example can be found here

private fun findOldestBookmarkTime(): Long {
        var bookmarkApiResponse = getBookmarks()
        var currentSearchpoint = step = bookmarkApiResponse.nextUrl.parameter

        while (true) {
            if (illusts.next_url != null) {
                // We are too high
                step /= 2
                currentSearchPoint -= step
                bookmarkApiResponse = getBookmarks(currentSearchPoint)
            } else if (illusts.artworks.isEmpty()) {
                // We are too low
                step /= 2
                currentSearchPoint += step
                bookmarkApiResponse = getBookmarks(currentSearchPoint)
            } else {
                // We have found it!
                break
            }
        }
        return currentSearchPoint
    }

Notes

• Using my personal Pixiv account, the oldest max_bookmark_id is found within four steps.
• I know the Pixiv website does allow you to jump pages when viewing bookmarks, however that uses a different API not available to me (I would need to acquire a PHPSESSID somehow).