Topic: Automatic Card Scanning and Entering

Update:
Hey folks! I've made some major changes to the way this script works. While the general idea is the same (using an LLM to extract data from cards and use that data to accurately populate a deckbox.org formatted CSV), the script now also integrates with Scryfall data for up-to-date card information and improved accuracy. I'd love to hear what you guys think and to know if it's been helpful at all.

Devs! Is anyone good with OCR? I'd love to skip the LLM completely, but with my limited programming knowledge I've been having trouble achieving any kind of accuracy when parsing the finer prints on Magic cards (important stuff like the set code and collector number). If this is an area you hold knowledge in, you're invited to make a pull request over on the github page. All the best!

Hi guys!

I just wanted to share a little project I've been working on in an effort to catch up on the cards I've neglected to enter into deckbox. I wanted to find a way to speed up the process of entering a large amount of cards at once without having to adjust minor details after the fact. What I landed on was a Python script that parses a folder full of pictures of your cards and then outputs a Deckbox formatted CSV for quick import into your collection. The workflow looks like this:

- Snap a picture of each card you want to enter into you inventory.  A quick one will do, we aren't looking to win any photo competitions.
- Place the pictures into the "card-pictures" folder in the scripts root folder.
- Open a terminal and enter "python main.py"

The script will resize each of your pictures, upload them to the OpenAI GPT-4o API for processing, and then fill output a CSV with all the relevant information. The thing that I really wanted automated for myself was the correct entry of the set code and card edition so that when I upload the CSV to deckbox I don't have anything left to tinker with. The script can, with surprisingly good accuracy if the lighting conditions are right in your photos, automatically identify and label your foils. I've setup command line args for all of the headings in the CSV if you'd like to do batches of certain "things" (foils, signed, condition, etc).

Currently, the barriers to entry are your own OpenAI API key and an entry level knowledge of command line use to use the script. I hope to simplify things sooner rather than later to provide an option for anyone not comfortable with the command line. For those of you comfortable using this kind of thing, I've love to hear if it worked out for you or if there is any specific features you'd like to see. You can grab the script over on my Github:

https://github.com/JackTheTripperr/MTG-Bulk-Scan

If anyone is interested and they need the extra help, just let me know and I'll write a quick how-to to get you up and running.

Cheers!

https://s10.gifyu.com/images/SrGbF.gif

Last edited by TheRippedTrojan (2024-08-02 23:17:53)

Re: Automatic Card Scanning and Entering

Sounds like a cool idea! What's the comparison set? Would newly added cards be available as soon as they're on Scryfall for instance?

Re: Automatic Card Scanning and Entering

Sure, here is the proofread and improved version of your text:

---

ZombieSleeve wrote:

Sounds like a cool idea! What's the comparison set? Would newly added cards be available as soon as they're on Scryfall for instance?

This is a good question and one I hadn't even considered while tinkering with this project. There isn't a "comparison set" per se, but the GPT-4o API has a knowledge cutoff date of October 2023. This might seem problematic for adding newer sets to your collection, but this is where large language models, especially multi-modal ones like GPT-4o, really shine.

As an experiment, I downloaded this picture and fed it into the script:

https://s12.gifyu.com/images/SrPW5.jpg

The results were quite interesting given the October 2023 cutoff. This is what was returned (apologies for the JSON format; the script's prompt requests the response to be formatted this way):

{
  "card_name": "Leonardo da Vinci",
  "set_code": "ACR",
  "set_name": "Alchemy Horizons: Adventures in the Forgotten Realms",
  "card_number": 20,
  "foil": "no"
}

Not perfect, but pretty good all things considered! More information can be returned, but this is all the current prompt in the script asks for. Even though the set isn't released , this would be good enough to accurately populate the CSV. I haven't done this yet, but I plan to modify the script so that it compares the set code from the card with the official deckbox sets and editions list (https://deckbox.org/editions). This would ensure the generated CSV isn't considered invalid due to minor discrepancies between official set names/codes and what's used on Deckbox. Another benefit of doing it this way is that the script would only need to return the card name, card number, and set code to generate an accurate CSV, meaning as soon as the set is added to Deckbox, it would function properly.

There is another OpenAI endpoint, the "Assistants" endpoint, that I'd eventually like to move this project over to. The Assistants endpoint would be better for this project for several reasons, but one of the more interesting ones is that the responses aren't limited by the knowledge cutoff of the language model. The Assistants endpoint allows several different "tools" to be called, one of which is the ability to search the internet. Sending the same prompt as above to the Assistants endpoint and giving it permission to search returns the following response:

{
  "card_name": "Leonardo da Vinci",
  "set_code": "ACR",
  "set_name": "Universes Beyond: Assassin's Creed",
  "card_number": "20",
  "foil": "no"
}

Completely accurate information for a set that isn't even out yet. And that's not all—you can prompt it to return a wealth of information. Your imagination is the limit:

{
  "card_details": {
    "card_name": "Leonardo da Vinci",
    "set_code": "ACR",
    "set_name": "Universes Beyond: Assassin's Creed",
    "card_number": 20,
    "foil": "yes",
    "card_type": "Legendary Creature",
    "subtype": "Human Artificer",
    "mana_cost": "{2}{U}",
    "converted_mana_cost": 3,
    "rarity": "Mythic Rare",
    "text": [
      "{3}{U}{U}: Until end of turn, Thopters you control have base power and toughness X/X, where X is the number of cards in your hand.",
      "{2}{U}, {T}: Draw a card, then discard a card. If the discarded card was an artifact card, exile it from your graveyard. If you do, create a token that’s a copy of it, except it’s a 0/2 Thopter artifact creature with flying in addition to its other types."
    ],
    "power": "3",
    "toughness": "3",
    "artist": "Wangjie Li",
    "legalities": {
      "Standard": "Not Legal",
      "Modern": "Legal",
      "Legacy": "Legal",
      "Vintage": "Legal",
      "Commander": "Legal",
      "Brawl": "Not Legal",
      "Pioneer": "Not Legal",
      "Historic": "Not Legal",
      "Pauper": "Not Legal",
      "Penny": "Not Legal",
      "Oathbreaker": "Legal"
    },
    "release_date": "2024-07-05",
    "printings": [
      {
        "type": "Nonfoil",
        "price": "11.50 USD"
      },
      {
        "type": "Foil",
        "price": "19.99 USD"
      },
      {
        "type": "Borderless",
        "price": "20.79 USD"
      },
      {
        "type": "Borderless Foil",
        "price": "23.00 USD"
      }
    ]
  },
  "gameplay_strategies_and_combos": {
    "gameplay_strategies": [
      "Thopter Token Generation: Leonardo da Vinci can turn your Thopter tokens into powerful threats based on the number of cards in your hand. This makes it ideal for decks that can draw a lot of cards quickly.",
      "Artifact Synergy: The card draw and discard ability can be used to create additional Thopter tokens, which synergizes well with other artifact-related cards."
    ],
    "possible_combos": [
      "With The Reality Chip: Equip The Reality Chip to Leonardo da Vinci to continuously manipulate the top of your deck, maximizing your hand size and thus the power and toughness of your Thopters.",
      "With Sai, Master Thopterist: Generate additional Thopter tokens whenever you cast artifact spells, further enhancing Leonardo da Vinci’s abilities.",
      "With Master Transmuter: Use Master Transmuter to cheat expensive artifacts into play, then discard them to create Thopter tokens using Leonardo da Vinci's ability."
    ],
    "deck_builds": [
      "Artifact Focused Deck: Build a deck around artifact synergies, including cards like Thopter Assembly, Wurmcoil Engine, and Etherium Spinner.",
      "Card Draw Mechanics: Include cards that draw you cards consistently, such as Curiosity Crafter and Skyscanner, to maintain a high hand size and boost your Thopters."
    ],
    "tips_and_tricks": [
      "Maximize Card Draw: Use cards that allow you to draw multiple cards to keep your hand size large, making your Thopters stronger.",
      "Protect Leonardo: Include protection spells and abilities, like Spellskite, to keep Leonardo da Vinci on the battlefield and functioning."
    ],
    "interesting_info": [
      "Leonardo da Vinci’s abilities are designed to make use of both card draw and artifact synergies, fitting well into decks that already focus on these strategies.",
      "The card's flavor ties in with Leonardo's historical association with invention and creation, making it a flavorful addition to any deck that enjoys thematic builds."
    ]
  }
}

Long story short, this project could be easily modified to fulfill nearly any purpose, and the information would be available at the same time (or possibly before) it's available on Scryfall. Anyhow, I'd like to move this project to the Assistants API in the future, but this is my first time working with Python and APIs, and the Assistants API is a bit more complicated to use than the basic GPT-4o one. Hopefully, I can do this sometime soon!

Ultimately, there are apps and websites out there that can scan cards faster and with less effort. I'm just having fun with this project because I wanted to explore using AI and Python, and this is what I landed on. Thanks for checking it out!

---

Re: Automatic Card Scanning and Entering

Hey,

This is very interesting indeed, nice investigations smile.

I did start a prototype for a card scanner, but not A.I. based, just image recognition stuff, which did not get too far. I do want to come back to it. That said, just using GPT through external APIs will probably be waay slower for now than some image recognition. Still, an avenue worth investigating, and I'm sure slowly the on-device A.I. capabilities for Android and iOS will get more appropriate for these usecases.

Re: Automatic Card Scanning and Entering

sebi wrote:

Hey,

This is very interesting indeed, nice investigations smile.

I did start a prototype for a card scanner, but not A.I. based, just image recognition stuff, which did not get too far. I do want to come back to it. That said, just using GPT through external APIs will probably be waay slower for now than some image recognition. Still, an avenue worth investigating, and I'm sure slowly the on-device A.I. capabilities for Android and iOS will get more appropriate for these usecases.

I agree. I actually had a good laugh with this morning after looking at scryfall and seeing that they have their own API…it’s clearly the better way of doing this. This is my first dip into “programming”, so there ought to be a few more surprises for me on the roads ahead.

For me, the biggest issue with entering cards has always been selecting the correct set/art…I’m a bit picky and I can’t stand when my cards are entered that way. Seeing the scryfall stuff, I’m thinking of doing something that just uses the set code and card number to populate the CSV with the relevant data from scryfall. I don’t know if opencv would be the best package for this or not, but I’m gonna look into it. The challenge so far is that the set code and card number is so damn small, it’s hard to have anything properly recognize it.

Re: Automatic Card Scanning and Entering

Hi guys! I made some big changes to this script that sees it integrating with the Scryfall API. Let me know if it's been useful for any of you! Cheers!