UPDATE: AI-POWERED APP NUMI

I wanted to share an update on Numi with you and E-Sylum readers.

I revamped my AI grading and identification app Numi to v1.33 to use the official American Numismatic Association's grading standards. My goal was to transform Numi to focus solely on technical grading, as technical grading is much more objective as compared to market grading.

Attached are some results for coins ranging from G-4 to PR-70.

As you can see, the results were quite off. With Numi getting increasingly accurate as the coin moved to a better condition.

Numi struggles quite a bit with recognizing wear on a coin. Interestingly enough, this was the exact issue Compugrade experienced back in the 90's. Even though their algorithms were completely different than today's AI Large Language Models.

 

 

 

 

 

 

Given Numi's failure to achieve consistent results, I am putting the project on ice until the next OpenAI GPT model is released. I'm also interested in testing Numi using Google's Gemini Ultra model once that releases sometime next year. Once they do, I will revisit these tests and see how much Numi has improved.

Observations & Lessons Learned

• Like what many online are reporting, GPT-4 has substantially degraded in performance in the past few months. I found myself constantly arguing with the AI to get it to follow most of my custom instructions.
• The model can follow custom instructions well for the first 8-10k tokens, but perception and logic fall off a cliff after that. Even with highly tailored instructions with clear steps to follow, Numi would veer off course when it had to process too much info. This was incredibly frustrating as GPT-4 Turbo claimed to have a token limit of 128k [i.e. ability to remember up to nearly 25k words].
• The vision model needs a major boost. The Optical Character Reader was great at picking up texts on coins. I rarely had it misread the text. But often it would fail to pick up the mint mark on more worn coins. The vision model could generally tell when a coin was more worn, but it failed to apply a correct grade.
• Numi was excellent at identifying non-ancient coins. And was quite often great at identifying tokens. But it struggled immensely with Ancient coins. The only ancient coins it could identify were coins in very high condition.

Overall Numi was a fun and exciting project. I got to apply my AI knowledge to my hobby and I'm excited to see how future models perform. AI is a force to be reckoned with and it's going to be an invaluable resource for researchers and those trying to learn more about their coins. I left this project feeling that AI technical grading would eventually work. But I now have my doubts that it can overtake market grading in a manner that would be widely adopted.

To read the earlier E-Sylum articles, see:
NUMI: AI-POWERED COIN IDENTIFICATION APP (https://www.coinbooks.org/v26/esylum_v26n47a10.html)
NUMI: HOW THE AI SAUSAGE IS MADE (https://www.coinbooks.org/v26/esylum_v26n47a11.html)

To read about the Brazilian Councilman writing laws with ChatGPT, see:
The City That's Trying to Replace Politicians With Computers (It's Working) (https://www.wsj.com/tech/politican-ramiro-rosario-artificial-intelligence-brazil-82ca338d)

 

	NBS (coinbooks.org) Web