Posts RSS Comments RSS Del.icio.us Digg Technorati Blinklist Furl reddit 63 Posts and Comments till now
This wordpress theme is downloaded from wordpress themes website.

Archive for December, 2014 (2014/12)

Sikuli (example Game of War bot automates simple tasks)

Overview

This post is some notes on my experience trying out Sikuli to write a simple Game of War bot.

Sikuli is a neat little tool that makes it super easy to write Python scripts for simple automation tasks that involve clicking on images.  Sikuli comes with an IDE that can take screen captures and shows you the screen capture as a thumbnail in your Python code viewer. I don’t know whether I imagine myself using this for anything serious, but it is at least a pretty cool concept and prototype.

You can also still edit your python code file (and png reference images) without the IDE.  The IDE generates an HTML file that it uses to display the code mixed with png thumbnails.

http://www.sikuli.org/

Here is a screen shot of the IDE:

image

Example setup and script for Game of War

Here is a code snippet that clicks the “Use” button in the Android game “Game of War”:

image

I wrote the script originally on my desktop using BlueStacks.  However, I’m on vacation now (visiting family) traveling with an old Windows laptop that doesn’t support BlueStacks (and even if it did, my screen resolution is smaller).  So I decided to tweak my script to work with VMLite VNC Server (Android) + TightVNC (desktop Windows).  Notice how the above code snippet has bIsLaptop, so it uses a lower resolution reference image.

Here’s a screen shot of TightVNC connected to my Nexus 7 with the resolution set to 50%.  DoUseLoop() clicks the Use button 500 times:

image

Concerns (Feedback to Sikuli development)

The IDE is a neat concept, but it’s not very robust.  The Windows version uses (ctrl+shift+z) instead of (ctrl+y) for redo.  A lot of the GUI menus you have to use the mouse (the keyboard doesn’t work).  There doesn’t seem to be an option to change the size of the thumbnails.

For a tool that seems to be all about convenience, a debugger would be nice, but I’d at least expect a simple option to have print() print to Sikuli’s console window. Instead, it prints stdout to the Message area in the IDE GUI.  It also prints debug messages to the Message area in the IDE GUI.  This is all well and good, except that it hides the IDE GUI when you run (and there is no option to avoid this).  Because the scripts run at the OS level (not the window level), this sort of makes sense, but it would be nice of there was an easy convenient option to keep a window open to see stdout debug (or status) messages.

I was also expecting a built-in shortcut to abort a script that you ran from the IDE, but I don’t see it.  Instead, I have to go to the IDE’s cmd.exe window and close that.  And when I close that, it doesn’t bring the IDE back, so I have to re-open the IDE window with runIDE.bat.  So there seems to be a lot of kludge even using this simple tool for the most obvious simple task.

The editor also has some other quirks.  For example, if I use (shift+left or right arrow key) to highlight an image in the text editor, it visually shows highlighting the entire line (even though when I do ctrl+x for cut I see that it did actually only highlight the image).  There’s a similar problem when I use (shift+home) or (shift+end) on a line of code with an image.

Conclusion

In any case, it works, and it is very easy to implement simple scripts such as this example of a simple script to play Game of War.  I actually feel like this might be a good way for a newbie (child or teenager) to get some very simple exposure to programming (learn through doing).  When I was in first or second or third grade (a long time ago with monochrome CRT monitors), we got to write simple Logo turtle programs, and our class even had a robot turtle that could draw things with a pencil.  This would of course go beyond that writing simple Python programs.

If someone is serious, they could of course run multiple instances of (Sikuli + BlueStacks).  Because Sikuli does image search and mouse click on the OS level, you would have to add logic to change which BlueStacks window is in front.  Or use a different tool that can do image search and mouse click directly to the window (instead of at the OS level).  Or just run one at a time, but have the script change between multiple accounts.  I’m not convinced Sikuli is the best tool to make a serious Game of War bot farm, but it’s at least a nifty tool.

Since it works at the OS level, it also means you can’t really use your computer while it’s running, unless you do it on a secondary computer or a virtual machine.  If the input (keyboard, mouse) and image detection worked at the window level (instead of the OS level), then you could run your script in the background while doing other tasks.

Legal Disclaimer: I don’t know if Game of War EULA has rules as against simple scripts like this. I do know that having multiple farm accounts is very common in the game, and that simple scripting would be an obvious way to automate many simple tedious tasks. I’ve also seen other posts about automating Game of War tasks including even a youtube video using Sikuli.  I also wouldn’t really call my example a “bot” as it only does simple tedious repetitive tasks based on the pixel output. “Game of War” has a lot of simple tedious repetitive button tapping (the company even calls itself “Machine Zone”).  Finally this post does not contain a code download, and it’s not really a tutorial.  The primary purpose of this post is just to share a quick example of one of the many small side projects I’ve done for fun outside of my day job, which shows (1) love of programming and (2) exposure to additional programming (knowledge, experiences, tools etc) outside of my day job (and college classes).  All that said, if anyone is offended by this post, I am glad to modify or delete it.

Scripting & Automation

I’m creating a new category to share some notes on some things I’ve done in terms of scripting & automation.  I do not consider myself a “test monkey” or an “automation specialist” or in any way affiliated with QA (quality assurance).  However, maybe 10% of my work for my day job has involved things related to scripting, automation, or regressions.  In addition to that, I am always on the lookout for manual tasks that I can automate (or simplify) to save time in the long-run (both at work and outside of work).  In some cases, I even prefer to automate something that doesn’t necessarily save time, because it can be less error prone.

Things I might post about include: AutoIt (AutoHotKey), Linux scripts, Windows scripts, Python, regressions…  But as always, I am not very dedicated to blogging (mainly because it’s a distraction from actually doing the stuff I’d post about) (and btw I’ve also already started doing weekly status notes for my day job), so no promises about what I will end up posting 🙂

For my day job, I primarily use C++ for actual development, and a mix of other things for scripting such as (simnow’s custom scripting language, python, Linux shell, Windows shell, AutoHotKey).  For simnow, we have our own simple custom scripting language (automation commands), and can easily share these scripts with users.

We automate more complex behavior (eg control flow) usually with Python (our regressions also use Python and XML).  I also write a lot of python scripts to automate tedious tasks (or to make sure I don’t do a human error).

I use Linux one-liners a lot on Linux, and also on Windows (cygwin is popular, but I’ve been using gnuwin32).  One example is to grep any sort of text files (such as simnow log files).  An example of Linux one-liners is to search log files (for debugging our simulator) (you can do a lot with just grep and awk).

I also like AutoHotKey (or AutoIt) mainly for how it integrates easily with Windows, and I can easily mix it with Python scripts.

Here’s a quick example of an awesome AutoHotKey script I use every day at work.  The background is that our IT department provides us file servers (NSF for Linux, SAMBA for Windows) where you can see the same files on Windows and Linux, and some file shares can also be seen from a web address.  For development, I typically use a local Windows computer in my cube, and I connect via NX to a farm of Linux machines (and I can submit jobs to LSF).

Because I use both Windows and Linux heavily in this way, I wrote an AutoHotKey script that activates when I do (ctrl+win+alt+c).  The script does the following…  It copies the clipboard contents to a network file (visible on both Windows and Linux).  It runs a python script that converts the path between (Windows, Linux, web) – if it’s one of the convertible paths.  For example, if I copy the path "/foo/bar/mypath" from my NX session, then do (ctrl+win+alt+c), then ctrl+V paste I will paste “\\winfoo\bar\mypath”.

For work related scripts I normally use Python, but I’ve used C# and Java and Objective C in side projects just for the exposure (and for college projects, back in the day).

Another example is the Torque 3D project (City Hero Defense) I did that used TorqueScript – you could expose Torque engine functionality to TorqueScript.  This sort of reminded me of how simnow models are written in C++, but they can expose simple automation commands for scripting.