Jump to content


Lorp - AdrianK's Local Replay Profiler

replay parser data

  • Please log in to reply
42 replies to this topic

AdrianK #1 Posted 06 January 2015 - 11:49 AM

    Major

  • Beta-Tester
  • 18215 battles
  • 2,539
  • [PBKAC] PBKAC
  • Member since:
    08-02-2012

Latest Update (13-Sept): Lorp v1.3 Released - WoT 9.10 Compatible

 

Download Lorp at Codeplex: https://lorp.codeplex.com/

 

Support content *might* be updated & extended in due course.

 

 

Intro 

 

I've built an application (that runs on your local machine) which will extract useful information from your replay files, provide some limited reporting, and let you export them to CSV (from which you can easily dump into Excel to do your own custom reporting.

 

Keep an eye on this OP for latest updates; then next few posts will contain supporting information (documentation, planned feature list, etc). :)

 

Download Lorp at Codeplex: https://lorp.codeplex.com/

 

About

 

This is a tool which runs locally on your PC (it doesn't need an internet connection to run).

It allows you to:
  1. Locate a notable replay by scanning your replay files and reporting back useful data.
  2. Let you export this data to CSV (and then into a tool like Excel where you can analyse the data.
  3. Provides some basic (but hopefully interesting & useful) reporting functionality.

Currently a decent range of information is extracted from the replay file but not everything.

 

Crawl Screen: 

 

Spoiler - Screen Shots

 

 

 

Known Issues

 

  • Fields not populated: "Battle Tier", "Platooned"
  • Credits earnt not populated for replay version ??? (December 2013)
  • Error sometimes occurs when crawling (usually against the current day).  Try crawling without any filters or date limits, cancel after a few files have been crawled and try again; otherwise close and restart.

 

Performance

 

  • Lorp uses parallel processing to crawl multiple replay files at once - based on the number of cores available, the more cores you have the faster Lorp will crawl the replays.
  • Performance is also dependent on disk-speed.  Running on an Lenovo X1 Carbon laptop (with 4 cores) I can process just over 30 replays a second - but that's with an SSD harddrive (no moving parts).  Based on (admittedly limited) information from others I estimate that using a more common spindle drive might lower that to around 10 replays a second.
  • As of Lorp v0.8 the Crawl screen includes both a Cancel button and provides an estimated time to complete; if you cancel the data already crawled will still be available to you.

 

Acknowledgements

 

Gort for use of his icon, see: http://www.forrestwalter.com/icons
Shout out to Stormtroop3r    amade    Maho__Nishizumi_ for providing old replays for testing.

The WoT Asia community, especially those active on the forums.


Edited by AdrianK, 13 September 2015 - 10:27 AM.

 Hope is not a strategy

Pray here to MM, RNG and Luck : http://mmrngluck.azurewebsites.net/


AdrianK #2 Posted 06 January 2015 - 11:49 AM

    Major

  • Beta-Tester
  • 18215 battles
  • 2,539
  • [PBKAC] PBKAC
  • Member since:
    08-02-2012

Content (mostly) current for Lorp version 1.0, schema 0.8.

How to Use 

See the User Guide (part of the application download).

 

Lorp should be intuitive enough to get started with on its own, but here's some help if you need it.

 

 

Spoiler - Installation Guide

 

Spoiler - Basic Running Instructions

 

Exporting

 

You can export data to a CSV file anywhere you see a data grid (like the one shown below).

 

Spoiler - Exporting

 

Using the Show Crawl Summary & Replay Detail functions

 

This feature is more for development support than practical use.

 

Spoiler

 

WoT Replay File Compatibility

 

The tool was initially developed and tested against WoT v9.4 & v9.5 replay files; recent work now provides proven support for version as old as 0.8.9, however the amount of test data available was a little limited so your mileage may vary.


Edited by AdrianK, 19 March 2015 - 11:49 AM.

 Hope is not a strategy

Pray here to MM, RNG and Luck : http://mmrngluck.azurewebsites.net/


AdrianK #3 Posted 06 January 2015 - 11:50 AM

    Major

  • Beta-Tester
  • 18215 battles
  • 2,539
  • [PBKAC] PBKAC
  • Member since:
    08-02-2012

Content (mostly) current for Lorp version 1.0, schema 0.8.

 

Current Features

Spoiler

 

Data Collected (v1.0) & Reported On

 

Spoiler

 

Planned Features

Spoiler

 

 

Change Log

 
(removed - see thread, I post new features in a post when the release is put up on CodePlex).

 

 

 


Edited by AdrianK, 19 March 2015 - 11:49 AM.

 Hope is not a strategy

Pray here to MM, RNG and Luck : http://mmrngluck.azurewebsites.net/


Ezz #4 Posted 06 January 2015 - 12:11 PM

    Major

  • Beta-Tester
  • 52286 battles
  • 31,417
  • [PBKAC] PBKAC
  • Member since:
    07-17-2012
Which replay versions does it support?

Who the [edited] are you? Get Spoofed!

>9000 cynicism brought to you by P2W, RIP Balance and the Cartoon Connection

Currently moderating your English speaking community : AALGMadibaCenturion_NZ, Elite911, Moonbase Patrol Copter 7

R. Pubbie: "why are all PBKAC players so rude, arrogant and nasty? and why do Mods favor them?"


AdrianK #5 Posted 06 January 2015 - 12:14 PM

    Major

  • Beta-Tester
  • 18215 battles
  • 2,539
  • [PBKAC] PBKAC
  • Member since:
    08-02-2012

View PostEzz, on 06 January 2015 - 04:11 PM, said:

Which replay versions does it support?

 

I'm still posting the content - (wife's calling me, so might be a delay...).


The tool was developed and tested against WoT v9.4 & v9.5 replay files, and appears at first glance to work with v9.3 replay files as well.

 


 Hope is not a strategy

Pray here to MM, RNG and Luck : http://mmrngluck.azurewebsites.net/


thommo_nz #6 Posted 06 January 2015 - 12:59 PM

    Sergeant

  • Council of Armored Forces
  • 43407 battles
  • 186
  • [NZAD] NZAD
  • Member since:
    10-03-2012

Nice work...

 


 

1. You might have to exempt temp.wotreplay from processes because of buggers like me who run it when WoT is open. WoT hooks that file.. :|

2. I have a number in there that show a status of 'Replay file incomplete'..  How are those 'counted'?


 


 



AdrianK #7 Posted 06 January 2015 - 03:33 PM

    Major

  • Beta-Tester
  • 18215 battles
  • 2,539
  • [PBKAC] PBKAC
  • Member since:
    08-02-2012

View Postthommo_nz, on 06 January 2015 - 04:59 PM, said:

Nice work...

 

1. You might have to exempt temp.wotreplay from processes because of buggers like me who run it when WoT is open. WoT hooks that file.. :|

2. I have a number in there that show a status of 'Replay file incomplete'..  How are those 'counted'?

 

Cheers.  yes, the tool will attempt to parse all files.  I plan to add functionality to help filter them out when it comes time to do analytics.


 Hope is not a strategy

Pray here to MM, RNG and Luck : http://mmrngluck.azurewebsites.net/


AdrianK #8 Posted 07 January 2015 - 09:57 AM

    Major

  • Beta-Tester
  • 18215 battles
  • 2,539
  • [PBKAC] PBKAC
  • Member since:
    08-02-2012

Does anyone have a Stronghold, Company and Team battle replay they can give me for testing?  If so please PM me.  Cheers.

 

Thanks Karl.


Edited by AdrianK, 07 January 2015 - 10:52 AM.

 Hope is not a strategy

Pray here to MM, RNG and Luck : http://mmrngluck.azurewebsites.net/


AdrianK #9 Posted 12 January 2015 - 01:56 AM

    Major

  • Beta-Tester
  • 18215 battles
  • 2,539
  • [PBKAC] PBKAC
  • Member since:
    08-02-2012

New Features (v0.2) :great:

 

  1. Added basic filters to the Data View screen.
  2. Vehicle Performance Report (with CSV export), see individual vehicle performance by map, battle-type and game-mode, includes damage ratio, credit profit and more.  This allows you to see common key stats for a given vehicle over different time slices - e.g. 1, 7, 30, 60 and 120 days (similar to forum signatures).
  3. Simple crawl options (get recent files by name or date filter).
  4. Properly skips temp.wotreplay files.

 

The Vehicle Performance UI:

 

 

 

 


 Hope is not a strategy

Pray here to MM, RNG and Luck : http://mmrngluck.azurewebsites.net/


AdrianK #10 Posted 14 January 2015 - 05:48 AM

    Major

  • Beta-Tester
  • 18215 battles
  • 2,539
  • [PBKAC] PBKAC
  • Member since:
    08-02-2012

New Features (v0.3)

 

  1. Fixed issue where the crawl attempts to parse the temp.wotreplay file still locked by the WoT client application.
  2. Added ability for user to define & save replay folder locations (as well as typing in ad-hoc folder addresses as per v0.1 & 0.2).
  3. Added ability to open the currently selected target replay folder from the UI.

Edited by AdrianK, 14 January 2015 - 06:31 PM.

 Hope is not a strategy

Pray here to MM, RNG and Luck : http://mmrngluck.azurewebsites.net/


AdrianK #11 Posted 17 January 2015 - 04:31 AM

    Major

  • Beta-Tester
  • 18215 battles
  • 2,539
  • [PBKAC] PBKAC
  • Member since:
    08-02-2012

New Features (v0.4)

 

Express release (my wifes back! I'll update the above content later).

 

  1. New data items:
    • Life Time (in seconds; how long you survived in the game)
    • Battle Duration (in seconds)
    • Survival Duration (as a percentage)
    • Lorp data schema version (schema = definition of what data is collected during the crawl).
  2. Ability to save replay data so you don't have to always crawl it every time.
  3. New screen for managing saved replay data, including the ability to merge data-sets together.
  4. Ability to save the (semi) raw replay data; this feature is to provide a degree of future proofing so that data-items not currently extracted by Lorp can be extracted by later releases (meaning you can delete your replay files if you wanted to).

Edited by AdrianK, 17 January 2015 - 04:32 AM.

 Hope is not a strategy

Pray here to MM, RNG and Luck : http://mmrngluck.azurewebsites.net/


AdrianK #12 Posted 20 January 2015 - 08:30 AM

    Major

  • Beta-Tester
  • 18215 battles
  • 2,539
  • [PBKAC] PBKAC
  • Member since:
    08-02-2012

I'm just working on trying to support older replay versions with the current functionality; so far I've got versions

  • 0.9.5 & 0.9.4 covered; 0.9.2 looks consistent with those (so that and 0.9.3 should be ok too).
  • 0.8.11 looks consistent with 0.9.5 as well (in terms of the file layout, so they should be ok too.
  • 0.8.10 & 0.8.9 is a bit murky though as the way the values are written is different (see below).  I'll be able to process these but will take a bit more work.

 

Block Quote - 0.9.5 format

 "achievementXP": 0, "deathReason": 0, "capturePoints": 0

 

Block Quote - 0.8.10 format

 'crits'p39I0sS'pierced'p40I1sg16I0sS'he_hits'p41I0ss

 

 

 

 


 Hope is not a strategy

Pray here to MM, RNG and Luck : http://mmrngluck.azurewebsites.net/


AdrianK #13 Posted 21 January 2015 - 03:00 AM

    Major

  • Beta-Tester
  • 18215 battles
  • 2,539
  • [PBKAC] PBKAC
  • Member since:
    08-02-2012

"Beta" vs Version 1.0

 

What's the difference? Well whilst still in beta expect that:

  1. The schema of the data extracted and saved may change (more than usual, and with less warning).
  2. The RawReplayInfo format to "future proof" things may change (it already is in v0.5 as I get to grips with older replay versions).  So don't delete your replay files just yet.

 


 Hope is not a strategy

Pray here to MM, RNG and Luck : http://mmrngluck.azurewebsites.net/


AdrianK #14 Posted 23 January 2015 - 02:22 PM

    Major

  • Beta-Tester
  • 18215 battles
  • 2,539
  • [PBKAC] PBKAC
  • Member since:
    08-02-2012
Fyi, parsing data from all versions going really well, just a matter of finding the time to finish that off.  I've made some adjustments to how I parse all files so need to test I haven't broken coverage of the newer versions of the game.  O hope to have an ETA (or trial version) ready this weekend.

 Hope is not a strategy

Pray here to MM, RNG and Luck : http://mmrngluck.azurewebsites.net/


AdrianK #15 Posted 26 January 2015 - 11:58 AM

    Major

  • Beta-Tester
  • 18215 battles
  • 2,539
  • [PBKAC] PBKAC
  • Member since:
    08-02-2012

Lorp v0.5 Beta Released

 

New features:

  1. Now supports older WoT replay file versions* including:
    • 0.9.5
    • 0.9.4
    • 0.9.3
    • 0.9.2
    • 0.9.1
    • 0.9.0
    • 0.8.11
    • 0.8.10
    • 0.8.9
    • Mystery version from February 2013 (version not recorded in replay file - as currently parsed).
  2. User Settings screen so you can add / remove favorite replay folder locations for easy crawling.
  3. Lorp now opens to the "Manage Data" screen where you can do stuff like merge sets of replay data into one file, for sharing, export or further analysis.
  4. Parsing process significantly refactored; you can now parse replays to (semi)-raw state and save them - this is partly to provide future-proofing.

 

Big shout-out to Stormtroop3r, Amade and Maho__Nishizumi_ for providing replays for testing. :honoring:

 

* Please note that due to limited test data (and time) your mileage in parsing old replay files may vary - let me know how you get on.


Edited by AdrianK, 26 January 2015 - 12:12 PM.

 Hope is not a strategy

Pray here to MM, RNG and Luck : http://mmrngluck.azurewebsites.net/


AdrianK #16 Posted 27 January 2015 - 01:30 PM

    Major

  • Beta-Tester
  • 18215 battles
  • 2,539
  • [PBKAC] PBKAC
  • Member since:
    08-02-2012

Freebie & Sneak Preview...

 

Q: "How can I find team mates (or adversaries) by name?"

 

Currently (v0.5) Lorp won't show you who all they players were in the replays, however, if you use one of the "save semi-raw replay data" crawl options you can then use Windows Explorer to find those games for you.  For example, below I've located all the replays that had PBKAC players in them; you can just as easily search for players by name.

 

 

 

Sneak Preview: Map Comparison Summary

 

One of the questions that been burning me for a while is knowing more about which maps I need to get better on.  Below is a screenie of a report currently in alpha; this will be more useful once I can factor in the tier of the vehicle played.  I also want to look at the break-down by spawn, but that'll most likely be in a later release.

 

 


 Hope is not a strategy

Pray here to MM, RNG and Luck : http://mmrngluck.azurewebsites.net/


AdrianK #17 Posted 29 January 2015 - 07:54 AM

    Major

  • Beta-Tester
  • 18215 battles
  • 2,539
  • [PBKAC] PBKAC
  • Member since:
    08-02-2012

Lorp v0.6 Beta Released

 

New Features:

  1. New Map Performance Reports - includes breakdown by game-mode and spawn.
  2. Vehicle & map performance reports now allow multiple selection.
  3. Made CSV export filenames more descriptive (when exporting off the vehicle & map performance reports).

 

Spoiler - Map Performance Comparison Report: By game-mode & spawn 

 

Next release will include adding vehicle tier & class to the core data set allowing you to do analysis on scenarios like "Are games with arty shorter"?


 Hope is not a strategy

Pray here to MM, RNG and Luck : http://mmrngluck.azurewebsites.net/


AdrianK #18 Posted 05 February 2015 - 03:41 AM

    Major

  • Beta-Tester
  • 18215 battles
  • 2,539
  • [PBKAC] PBKAC
  • Member since:
    08-02-2012

Lorp v0.7 Preview

 

Update - I forgot to mention, I've introduced some performance buffs - the crawl speed is almost doubled (even with the additional team & reference data stuff - see below).  Processing of 1200 replays now takes about 35 seconds, was over a minute (that's approx 1 replay every 30th of a second, assuring I'm not too bad@maths).  For the geeks, I'm now using .net 4.5's  Parallel.ForEach.

 

I've pulled in vehicle reference data now, so you'll (eventually) be able to see performance by tier (etc).  It also means I can provide team break-down by vehicle type, etc.  You'll also be able to search (in excel, once you export) for players by name (I haven't included clan names but could add that).

 

Here's what the new data looks like in the data view at the moment. 

 

 

Here's what the team list's look like - tier, class, common vehicle name & player name, sorted by tier: 

 

8 TD JgTig.8 SB360 / 
8 TD JPanther II AdrianK / 
8 Hvy T34 Arex_Strong / 
8 SPG G.W. Tiger P artworkcafe / 
8 Med Panther II SiIenT_WarriOR / 
8 Hvy FCM 50 t PewBooMpeW / 
7 TD SU-152 Ghilgamesh / 
7 Hvy T29 Mnoplive / 
7 Hvy Tiger I 93gyer / 
7 Lt AMX 13 75 SFqymsun / 
7 TD JPanther rae_sia / 
6 TD SU-100 kpj / 
6 Hvy ARL 44 KerryTank / 
-1 Not found uk-GB75_AT7 han77718 / 
-1 Not found uk-GB21_Cromwell NiteTribe / 

 

 

You'll note my reference data collection isn't complete (the "-1's" - I'm working on that) but the good news is that the reference data will be stored as a CSV file with the application so it'll be very easy for you to update as needed.

 


Edited by AdrianK, 05 February 2015 - 06:28 AM.

 Hope is not a strategy

Pray here to MM, RNG and Luck : http://mmrngluck.azurewebsites.net/


AdrianK #19 Posted 10 February 2015 - 08:02 AM

    Major

  • Beta-Tester
  • 18215 battles
  • 2,539
  • [PBKAC] PBKAC
  • Member since:
    08-02-2012

Lorp v0.7 Released

 

  • Faster crawl processing (performance on my dev machine has effectively doubled, now processing approx 30 replays a second).
  • Added SPG Impact reports (2) to new "Misc Reports" screen.
  • Added quick visual to Vehicle Performance screen.
  • General polishing & usability enhancements.

 

And other stuff I've forgotten.


 Hope is not a strategy

Pray here to MM, RNG and Luck : http://mmrngluck.azurewebsites.net/


ouchthathurts #20 Posted 10 February 2015 - 04:21 PM

    Major

  • Member
  • 22728 battles
  • 3,868
  • [PBKAC] PBKAC
  • Member since:
    10-28-2012

Well, this is frustrating. While trying to debug some problems with my WoT installation last month, I now notice that I accidentally deleted almost three years of replay files going back to 0.8.3. :-(

 

If you need data from 8.4 through 8.7 for testing, however, I do have my NA server replays from those versions (about 2 GB worth) on my DropBox account; PM me if you'd like access to download some or all of it.


Edited by ouchthathurts, 10 February 2015 - 04:31 PM.

All pre-9.5 tier I through VI tanks played and elited.





2 user(s) are reading this topic

0 members, 2 guests, 0 anonymous users