Support this project now at Patreon, or donate directly via PayPal:
The development plan, now that we have reliable data collection, is to start to share the tool; but more importantly to share the data. I will be publishing clearer details of the XML format being used, however, it is a node only based text format, at present a commodity looks like this:
I plan to wrap this basic format into an outer "Commodities" node, with the date & time included, and make that publically available.
However, I do not plan to generate that whole list - especially as the data set grows - I plan to generate that list maybe weekly at most, as I am going to be driving this for free*.
To recap the current process for capturing the information, you play the game, and run the image processing application, then from within the game client - without tabbing to the image processor, you just press keys - you can send commands to the image processor to capture images of the navigation and commodities screen, to start them capturing the character information with tesseract:
Armed with your local data, you then run the separate windows application - and yes you do have to be able to tab to this application - to perform searches of your collected data store, which looks like this:
If you opt into sharing your data you are helping the project, at the most basic level, providing your data you will help yourself and other players! So by sharing your local data store it gets uploaded - periodically - to the server.
Once you run the trade analyser, when you've been sharing your data, it will download the latest public list and start to use this, if you are a special supporter, sponsor or donor this is where your live cycling of the data will take place, the trade tool will automatically authenticate with the server, and start to receive the updates from all other players faster than the general list.
Of course, you will be able to write your own tool to access this same data, and be able to call the server to download the current public list of commodities and use that in your tools, other sites and projects are welcome to use this data as we collect it. However, I do ask that you share, and help this project collect data in return.
I will be generating the special patron, sponsored, active list at least daily, if not hourly, so anyone supporting the project through Patreon, or through a sufficiently high single donation will be given access levels giving them near live, or as I say at least daily updates of the information from the project.
This won't does not affect your local data store, if you collect data live, then it is your data, and you opt into sharing that information.
As the project matures I also hope to be capturing system connectivity data, this is where other information - such as your ship's current jump range - may be required, so help the server rationalise the data.
The server will also afford a level of checking and sanitisation to the data, so signing up to help the project you will receive a more accurate data set, not only for the commodities names, but the prices as well as the supply & demand.
What do I mean by sanitised data? Well, the OCR is not perfect, as a side effect of either over-training, or under-training, it makes mistakes. It also makes mistakes when the background behind your commodity screen changes - this can be because you were using the capturer whilst in the hangar, or whilst just on the surface of the landing pad. These variations affect the isolation and interpretation being carried out.
Here is a good example:
Clearly the reader got it wrong on at least one pass, and created a new erroneous commodity, or sometimes it just plain gets it wrong:
You can spot these kinds of mistakes in the other screen shots I've already published:
The server however will be able to sanitise this data, that is correct spelling mistakes, or ask the master list (or myself) to confirm what is meant by a new name or a miss-spelled name. Helping correct this data.
Your local data store will not do this and such mistakes will just sit there, until removed or updated by your sharing your data and receiving the new ratified list of commodities.
As well as mistakes with the name, sometimes it also makes mistakes with the values being read, this can lead to interesting problems. I did once set off on a 4 system flight loaded with (you know I forget what) but it had a profit of 800,000 credits!!!! WHAO!
Only when I arrived did I find out I'd made a loss, because I'd bought these things at something like 2,000 units, and the OCR had mixed up the 1877 of the destination with 7811, vasting increasing the profit margin the tools thought I would get.
This problem has been reduced, but again depending on the background at the moment the commodities or captured, of if they are a "good" screenshot or not it can happen that mistakes are made.
By sharing the data however, there will be multiple, perhaps even tens or hundreds of imprints of the same commodity at the same station coming in, the server will be able to average or at least rationalise these values and filter out wildly high values!
This is data sanitation, filtering out the outrageous mistakes we as humans would spot instantly, ornot even make as we can read the screen perfectly well.
Of course, running such servers is going to be costly, and so to share we do ask you help support the project.... Over at patreon you will find our page.
No comments:
Post a Comment