[Other] PHP Dota Replay Parser (CDP)

Seven · 06-11-2009, 06:59 PM

CDP - CTS Dota Parser

Preface
I've been working on and toying around with a PHP dota replay parser I whipped together using *Reshine and *esby's xml dota data as a starting point.

I've tried to keep the code as clean as possible, however since the core of the parser's been worked on by several programmers before me, it's not exactly the best example of clean and structured code out there.

Still I believe the code is comprehensive enough and at the very least the w3g parsing procedures and Dota ID data can be a useful reference for anyone working on their own Dota replay parser.

A brief on security
Be aware that the purpose of this package is to provide a PHP dota parser to developers for use in their own projects. While the bundled upload, storage and replay display interfaces are useful to an extent they are in no way intended for a production environment.
When implementing your own replay handling interfaces make sure to account for security issues such as XSS, CSRF, SQL Injections when you roll out your more efficient storage mechanism and so on. The parser does not strip malicious tags from strings on its own. So make sure to properly filter strings from user submitted data, even ones parsed from a replay.

An easy exploit that comes to mind is a malicious user uploading a replay with an XSS payload hidden in chat logs. The attacker could end a string (") and include a <script> thus exploiting your visitor's browser data and sessions.

A good primer on web security is Google Gruyere. Another useful learning application is DVWA.

* see credits

Update 1.4.4 (6.1.2012) : 6.73 data and images
Update 1.4.3 (14.5.2011) : 6.71 & 6.72 data and images
Update 1.4.2 (26.12.2010) : 6.70 data and images
Update 1.4.1 (10.11.2010) : 6.69 data and images
Update 1.4 (16.8.2010) : 6.68, Switch & Debugging
Update 1.3.1 (1.4.2010) : 6.67c, 6.66 data and images
Update 1.3 (7.1.2010) : 6.65, SP & Debugging
Update 1.2.2 (27.10.2009) : Debugging skilling & 6.64
Update 1.2.1 (8.10.2009) : Handling morphers
Update 1.2 (5.10.2009) : Hero - Player - Skill handling
Update 1.1.3 (10.9.2009) : Ban / Pick duplication bug.
Update 1.1.2 (7.7.2009) : 64 Bit memory / 6.61.
Update 1.1.1 (19.6.2009) : Decompression tweak.
Update 1.1 (12.6.2009) : Added 6.60 support.

I can't seem to attach a new file, so use the Download links bellow for versions 1.4.4 and above.

Dota 6.74 Map data: (download) (CDP 1.4.4 ships with 6.73 data)

Download: CDP Latest (Zipped / 1.4.4) | Mirror
Sample: Sample Parsed Replay

Patches
Patch 1.1.1 to 1.1.2
Patch 1.1.2 to 1.1.3
Patch 1.1.3 to 1.2
Patch 1.2 to 1.2.1 ( 16 kb )
Patch 1.2.1 to 1.2.2 ( 74 kb )
Patch 1.2.2 to 1.3 ( 469 kb )
Patch 1.3 to 1.3.1 ( 195 kb )
Patch 1.3.1 to 1.4 ( 274 kb )
Patch 1.4 to 1.4.1 ( 182 kb )
Patch 1.4.1 to 1.4.2 ( 407 kb )
Patch 1.4.2 to 1.4.3 ( 112 kb )
Patch 1.4.3 to 1.4.4 | Mirror ( 263 kb )

About

CDP CTS Dota Parser is an open source Dota Replay parser written in PHP,
meant to be used with WC3 DoTA replays of version 6.59 and higher.

Quick Feature List

Supports CM mode and properly lists bans and picks
Supports swapping
Supports switching
Supports shuffle player mode
Displays end game statistics that include
- Hero Kills / Deaths / Assists
- Creep Kills / Denies
- Neutrals killed
- End game gold
- End game Inventory
- POTM Arrow Accuracy
- Pudge Hook Accuracy
Tries to automatically determine the winning side.
Categorizes player’s actions and calculates his APM excluding picking time.
Generates time ordered lists of player’s obtained items and learned skills.
Displays colored chat
XML database of Items, Skills and Heroes.
Easy storing and restoring of parsed replay data with php serialization

Installation

Extract the files.
Ensure the structure is as follows:
- ./*.php *.css - All the php / css files
- ./maps/*.xml - All the XML data files
- ./images - All the icons
- ./replays - Folder for storing replays and replay info
Make sure the ./replays folder is writable.

Usage

Use the ./upload_reply.php file to upload replays or
write your own interface, using the included
upload_replay.php file as a guide.

All the replays are parsed upon the first view and a
serialized representation of the replay is stored as
a REPLAY_ID.txt file. Upon further viewing the replay
data is only obtained from the serialized object.

All the replays are saved as unix_timestamp.w3g files
while conserving their original name in the $replay->extra
variable, making it possible to let the user download
the replay with the uploader’s original filename.

Credits

Julas - Original wc3 php parser
rush4hire - Dota port of Jula’s parser
esby - 6.56 XML Structure / Data
Tedi Rachmadi - Reshine

Christian · 06-11-2009, 07:13 PM

Seven you have done a great job - this is the first php replay parser with trimed code. A few weeks ago I started to integrate a replay parser into my clan website. I have not had many experience with w3g, so I used esby's (sure this was a version based on julas parser etc.) existing replay parser and try to adapt it. The problem was the whole code was a real chaos, because it supports all previous versions of Warcraft and so on. At least I was successful but it takes me much time and the code is not better structured now.
Your source is very nice - I think I will use this code now because it make it much easier for me to make it adapting to our website. Nice done

Terrens · 06-11-2009, 10:42 PM

Looks fine

I will test it soon on my clansite.
Thanks for good job

Dub · 06-11-2009, 11:11 PM

BatRider has no items, yet they are shown under the items tab as having items. You may need to edit the xml file for the new version and fix the codes. I didn't really look close enough but thats probably what it is.

Other then that nice job. Maybe split the bans into scourge and sentinel? It also doesn't say who won.

- Steve

Seven · 06-11-2009, 11:19 PM

Hey, hope you won't have too much of a hassle getting it working for your clan sites.

As far as your comment goes Dub, the Batrider player left before the game ended, and his teammates proceeded to sell his items, thus his inventory is empty. The reason you see the item "build up" in the "Items" tab, is because those are the items the player actually bought from the Shops through out the game, while the final inventory is populated at the end of the game.

As far as the ban splits go, there's a "pink" and "blue" underline border which indicates the Player that banned or picked the hero. While that may not be the best way to show it, I didn't come up with a better way so far, but the bans are of course split in the code itself and it's just a matter of choosing how the data is presented in the final output.

TheSaw · 06-12-2009, 10:09 AM

The light colors of the names are a bit hard on the eye, but the parsing is nice... I have to say the best one around, because all the other parsers are lacking some serious updates and polishing.

TeT · 06-12-2009, 11:56 AM

This should be included on this website for attached replays !

CryoNine · 06-12-2009, 02:09 PM

Amazing job! Will definitely be using this.

sprion · 06-12-2009, 03:09 PM

impressive

Apocalypse · 06-12-2009, 05:16 PM

Super imba!!!!!!!!!!!!!!

Nameless · 06-12-2009, 05:29 PM

Can you post a screeny please?

SpeakerOfDead · 06-12-2009, 07:08 PM

Really nice script m8.

Amornthep · 06-12-2009, 07:19 PM

Thanks for the parser. I will be using this from now.

Christian · 06-12-2009, 08:24 PM

@Nameless

There is not much for a screen. Have a look at the example replay to see the result. The other thing is just a formular to upload the replay.

DaTaBASE · 06-14-2009, 06:01 PM

Man awesome,maybe i will use it on my site,anyway, very gj,so unselfish

dev0 · 06-17-2009, 02:23 PM

What is the licence on this? If I recall Julas parser was GPL licenced, or? Due to the unfectious nature of GPL this must also be GPL licenced, or?

Christian · 06-17-2009, 02:35 PM

i think so^^

Seven · 06-17-2009, 02:38 PM

Yea Copyleft / GPL it up.

dev0 · 06-17-2009, 02:54 PM

I've implemented your parser on our Clansphere powered website and released it as a part of a clansphere module implementation:
Clansphere Module
Demo on our (german) website

Thanks alot for your great work!

Seven · 06-17-2009, 02:59 PM

Looking good, hope it serves you well.

06-11-2009, 07:13 PM	#2
Christian Member Join Date: Jun 2009 Location: Germany Posts: 190	Seven you have done a great job - this is the first php replay parser with trimed code. A few weeks ago I started to integrate a replay parser into my clan website. I have not had many experience with w3g, so I used esby's (sure this was a version based on julas parser etc.) existing replay parser and try to adapt it. The problem was the whole code was a real chaos, because it supports all previous versions of Warcraft and so on. At least I was successful but it takes me much time and the code is not better structured now. Your source is very nice - I think I will use this code now because it make it much easier for me to make it adapting to our website. Nice done

Last edited by Christian; 06-11-2009 at 07:17 PM.

06-11-2009, 10:42 PM	#3
Terrens Member Join Date: Jun 2009 Location: USSR Posts: 72	Looks fine I will test it soon on my clansite. Thanks for good job __________________ We play games, no games play us. (c) Bandits` style.

06-11-2009, 11:11 PM	#4
Dub Member Join Date: Jun 2009 Posts: 29	BatRider has no items, yet they are shown under the items tab as having items. You may need to edit the xml file for the new version and fix the codes. I didn't really look close enough but thats probably what it is. Other then that nice job. Maybe split the bans into scourge and sentinel? It also doesn't say who won. - Steve __________________ <?php while($time != "23:59-07:00") { play_dota(); } ?>

Last edited by Dub; 06-11-2009 at 11:16 PM.

06-11-2009, 11:19 PM	#5
Seven Member Join Date: Jun 2009 Posts: 61	Hey, hope you won't have too much of a hassle getting it working for your clan sites. As far as your comment goes Dub, the Batrider player left before the game ended, and his teammates proceeded to sell his items, thus his inventory is empty. The reason you see the item "build up" in the "Items" tab, is because those are the items the player actually bought from the Shops through out the game, while the final inventory is populated at the end of the game. As far as the ban splits go, there's a "pink" and "blue" underline border which indicates the Player that banned or picked the hero. While that may not be the best way to show it, I didn't come up with a better way so far, but the bans are of course split in the code itself and it's just a matter of choosing how the data is presented in the final output. __________________ "An eye for an eye makes everybody blind."

06-12-2009, 11:56 AM	#7
TeT Member Join Date: Jun 2009 Posts: 176	This should be included on this website for attached replays ! __________________ Guide : How to add rune Sounds

06-12-2009, 10:09 AM	#6
TheSaw Member Join Date: Jun 2009 Posts: 24	The light colors of the names are a bit hard on the eye, but the parsing is nice... I have to say the best one around, because all the other parsers are lacking some serious updates and polishing.

06-12-2009, 02:09 PM	#8
CryoNine Member Join Date: Jun 2009 Location: Buffalo, NY Posts: 58	Amazing job! Will definitely be using this.

06-12-2009, 03:09 PM	#9
sprion Member Join Date: Jun 2009 Posts: 29	impressive __________________

Last edited by sprion; 06-12-2009 at 03:21 PM.

06-12-2009, 05:16 PM	#10
Apocalypse Member Join Date: Jun 2009 Location: Spiritual Black Dimensions Posts: 107	Super imba!!!!!!!!!!!!!! __________________

06-12-2009, 05:29 PM	#11
Nameless Member Join Date: Jun 2009 Location: Japan/Philippines Posts: 1,068	Can you post a screeny please? __________________ PinoyGG - Pinoy Gaming & Tech Community PinoyGG Group - PinoyGG Page

06-12-2009, 07:08 PM	#12
SpeakerOfDead Member Join Date: Jun 2009 Location: P(r)oland, Warsaw Posts: 11	Really nice script m8. __________________

06-12-2009, 07:19 PM	#13
Amornthep Member Join Date: Jun 2009 Location: Singapore Posts: 33	Thanks for the parser. I will be using this from now.

06-12-2009, 08:24 PM	#14
Christian Member Join Date: Jun 2009 Location: Germany Posts: 190	@Nameless There is not much for a screen. Have a look at the example replay to see the result. The other thing is just a formular to upload the replay.

06-14-2009, 06:01 PM	#15
DaTaBASE Member Join Date: Jun 2009 Posts: 83	Man awesome,maybe i will use it on my site,anyway, very gj,so unselfish

06-17-2009, 02:23 PM	#16
dev0 Member Join Date: Jun 2009 Posts: 4	What is the licence on this? If I recall Julas parser was GPL licenced, or? Due to the unfectious nature of GPL this must also be GPL licenced, or?

06-17-2009, 02:35 PM	#17
Christian Member Join Date: Jun 2009 Location: Germany Posts: 190	i think so^^

06-17-2009, 02:38 PM	#18
Seven Member Join Date: Jun 2009 Posts: 61	Yea Copyleft / GPL it up. __________________ "An eye for an eye makes everybody blind."

06-17-2009, 02:54 PM	#19
dev0 Member Join Date: Jun 2009 Posts: 4	I've implemented your parser on our Clansphere powered website and released it as a part of a clansphere module implementation: Clansphere Module Demo on our (german) website Thanks alot for your great work!

06-17-2009, 02:59 PM	#20
Seven Member Join Date: Jun 2009 Posts: 61	Looking good, hope it serves you well. __________________ "An eye for an eye makes everybody blind."