The goal of this project was to predict preterm birth (a birth which takes place between 20 and 37 completed weeks of gestation) based on medical, demographic, and lifestyle data. In a team of three, we compared the predictive performance of various classification techniques (such as Naive Bayes, Decision trees, SVM, logistic regression, and associative classifiers).

• Project Context: Mojdeh Jalali Heravi, Amit Satsangi, and I worked on this project as part of our Mining Health and Medical Data course.
• Dataset The original dataset was collected by Northern and Central Alberta Perinatal Outreach Program between 1992 and 2003. The dataset contains maternal and newborn data for 243948 cases, including 21193 preterm cases.
• Mining Tools We used WEKA for most of the classification tasks. Other tools/algorithms include: SPSS, STUCCO, and Eclat.

A detailed overview of the applications of data mining in administrative, clinical, research, and educational aspects of Health Informatics. The current and potential applications of various data mining techniques in Health Informatics are illustrated through a series of case studies from published literature.

An investigation of using additional movie information from the IMDb to make better recommender systems for the Netflix Prize challenge. In a team of four, we developed a number of Hybrid Recommender Systems, and were able to slightly improve the predictions submitted by the University of Alberta "Reel Ingenuity" team.

• Project Context: Babak Bostan, Mojdeh Jalali-Heravi, Amin Jorati, and I worked on this project as part of our Topics in Machine Learning course.
• Recommender Systems Recommender systems suggest new items to users based on their activity history (such as previous purchases, items viewed, or ratings). In the case of Netflix, a movie recommender system suggests new movies to users based on their rental history.
• About Netflix Prize In October 2006, Netflix released a large set of customer ratings, and offered a prize of one million dollars to anyone who can improve their current prediction accuracy by 10%. This has turned out to be a difficult challenge, and after more than a year, no team has yet met their required accuracy.
• Datasets The Netflix Prize training set contains one hundred million ratings of more than half a million users rating 18000 movies. In addition to the Netfix dataset, we extracted various features such as genre, actor and director for each movie from IMDb. Among more than 430,000 media titles in downloadable text files from IMDb, we were able to match about 8800 movies in the Netflix dataset using title and release year.
• Results Our results indicate that combining hybrid methods based on extra movie information has the potential to improve the accuracy of current collaborative methods in the Netflix challenge. (See the Project Report for a detailed description of our approaches and the results)

A method for ranking academic conferences, articles, and authors by applying PageRank (the algorithm that Google uses to rank its search results) on the citation network.

The goal of my Masters thesis is to build an agent that can learn to play arbitrary Atari 2600 games. The agent receives screenshots from the screen, and is able to move a virtual joystick. The agent should learn to play a new game by moving around the game and learning desirable and undesirable states. We are currently experimenting with computer vision techniques and Reinforcement Learning methods with promising results.

You can watch a demo video of the gradient-descent Sarsa Lambda algorithm learning to play the game Freeway on Youtube.

An AI agent that learns to play a simplified Real-Time Strategy game (in particular, a simplified version of Lunticus).

The AI agent finds a plan that builds the required buildings and saves enough money to beat the opponent. To do this, the game space is treated as a search graph, and I apply the Dijkstra algorithm to find the optimal path to a goal state.

You can watch the AI agent (the left player) execute a sample path against its opponent here (2D textures are turned off, so it may be hard to distinguish buildings).

Reimplementation of Armor Games: Sea of Fire Real-Time Strategy game in Python. It uses the Opioid2D game framework, which in turn uses OpenGL for rendering. The game graphics are from the GPL strategy game Bos Wars .

The current implementation has no AI player, but if you feel like playing for both sides, it is fully functional.

A simple strategy game developed in Matlab to demonstrate the application of machine learning and probabilistic modeling techniques in game AI. The AI player uses a Sequential Monte Carlo algorithm to estimate the opponent's strategy in different game settings. It then applies a policy search algorithm to choose its own strategy based on the estimate of the opponent's strategy.

A Space-Invader style arcade game written in C++ using Open-GL. Using Evolutionary algorithm, the aliens learn how to survive in the game environment as the game goes on. Usually, within about five minutes of playing, the neural networks of the alien brains evolve to avoid the gun and hitting other aliens.

• Project Context: My cousin and I worked on this project -mainly as a pastime- while she visited Vancouver in August 2005.
• Original Idea was taken from Mat Buckland's AI Techniques for Game Programming book. Trying to take his idea a step further, we designed a more challenging game environment and rewrote the game from scratch. Finally, by experimenting with different neural network structures and altering the evolutionary algorithm, our implementation produced much improved results.
• Game Rules: The player's objective is to shoot and blow up the aliens. The aliens' objective is to avoid the gun and steer clear of other aliens while staying within the world boundaries.
• Evolutionary Algorithm Description: Here is a brief overview of the algorithm:
  • Alien Brain: Each alien has a three layer neural network which controls its actions. The neural network inputs are: the position of the gun and the relative speed of the near bye aliens.
  • Alien Population: Initially, the alien population is initialized with 10 aliens with random neural-nets. As the game continues and aliens are blown up, these newly destroyed aliens are put back in the pool. If the size of the pool grows larger than 100, some of the worst 'fit' aliens are removed from the population.
  • Alien Reproduction: To replace a destroyed alien two aliens are selected from the pool. Using -Crossover and Mutation- a new alien is then generated.
  • Natural Selection: The probability that an alien is selected to reproduce is determined by its fitness. The more 'fit' an alien is, the more chance it has to reproduce.
  • Alien Fitness: The fitness of an alien is defined as the time it survived in the game environment. The longer an alien survives -i.e. is not blown up by the gun or by hitting another alien- the more 'fit' it is to reproduce
• Convergence: In our more than 100 trials of the game, the evolutionary algorithm always converged in less than 10 minutes. The aliens always learned to avoid the gun without bumping into each other. You can watch the demo video to see a typical convergence.

An attempt to learn to play Texas Holdem Poker using Reinforcement Learning. The program is written in C++ and it uses Q-lambda algorithm. The agent does learn some aspects of poker, but generally fails to converge to an optimal policy.

• Project Context: Ian Macdonald, David Matheson, and I implemented this program for our 'CPSC 422: Intelligent Systems' course project in March 2005.
• State Representation: The state space in poker is extremely large. Over 2 million states would be needed just to represent the possible combinations for a single 5 card poker hand. We came up with a new state representation which is magnificently smaller and still captures the essential characteristics of the game. (You can read our project report for detailed description of our state representation)
• Reinforcement Learning: The agent plays multiple games with another agent (human or computer player). At the end of each game, the amount of money won/lost is used to reward/punish all the states that were responsible for the agent to end up in its final state.
• Convergence: In our trials, while the agent did learn some aspects of poker -for instance it learned to Raise or Call when it has a good hand in the last round- it failed to converge to an optimal policy. Also, while it tend to play reasonably well against a human player, it played very poorly against a bluffer agent (an agent that always raises, unless raising is not possible, in which case it calls). We explore some of the possible reasons of this failure in our report.
• Happy Ending: While the agent didn't converge to an optimal policy for Texas Holdem Poker, it did earn us a very good mark for the project! :)

An attempt to solve the Grid World problem with an Evolutionary Computation approach, instead of the original Reinforcement Learning approach.

• Project Context: This was a personal project that I worked on in summer 2005. I was curious of how well an evolutionary approach would compare to the RL approach.
• The Main Idea is that in the optimal policy, the neighboring cells are very likely to have the same optimal action (i.e. if in cell (3, 2) the best action is to move right, it is very likely that in cell (3, 3) the best action is also moving right). Therefore, instead of finding the optimal action for each individual cell, we can think of the problem as finding blocks (neighborhoods) with the same and optimal action.
• Search Space: I use quadtrees to partition the grid world, by recursively subdividing it into four quadrants. Each leaf of the quadtree represents a block of neighboring cells with the same optimal action. The problem is now searching over all possible policy quadtrees to find the optimal one. I use an evolutionary algorithm to search this search space.
• Results: In my trials, the algorithm does converge to a near optimal policy, but the convergence rate is significantly slower than the Reinforcement Learning approach. Overall, while the result wasn't satisfactory, it was a fun project and I learned a lot about implementing evolutionary algorithms.

An AI agent for playing Othello boardgame using Min/Max tree algorithm with Alpha-Beta pruning. The game is implemented in SML and includes a command line interface, so the user can play against the AI.

• Project Context: Ian Macdonald, David Matheson, and I implemented this program for our 'CPSC 312: Functional and Logic Programming' course project in November 2004.

A real-time simulation of toppling dominos written in C++ using Open-GL. The user provides a pattern in an image file and the program puts a number of dominos on the desired curve. Using rigid body dynamics simulation, the program then generates a domino effect.

• Project Context: I implemented this program for my 'CPSC 426: Computer Animation' course project in fall 2005.
• Features: Rigid body dynamics simulation, Vertex-Face and Edge-Edge collision detection, collision response, collision resolution (*buggy), resting contact with friction on the ground. Unfortunately, I didn't have much time for this project so the collision resolution does not work very well and sometimes the dominos penetrate each other.

A non-real-time simulation of a mud-like particle system with frictional collisions. It was an assignment in 'CPSC 426: Computer Animation' course.
Acknowledgement: Some of the code, in particular the code to create and render blobbies from particles, was provided by the instructor.

A web-enabled network discovery and analysis tool developed in C and Python. It is capable of mapping and analyzing the routes on which data packets flow from a source machine to multiple destination host machines.

• Project Context: Iulian Radu, Matthew Sniatynski, and I implemented this program for our 'CPSC 417: Computer Communications' course project in June 2005.
• Summary: Target IP addresses are provided as input via the web interface. Using traceroute and Graphviz, the program generates a graph of the network paths as seen by the source machine. Output is generated in PNG and NS script file format, allowing further analysis and simulation using the Network Simulator / Animator utilities.

In summer 2003, while working for Douglas College Learning Centre, I created this handout to help students who were taking pre-calculus math courses develop a better understanding of mathematical functions.