Side Projects
During my spare time, I also implemented lots of tools/algorithms/papers that I found interesting, all of which are open-source.
 | Pytorch implementation of World Modelscode /This is a Pytorch implementation of Google World Models. |
| | Tensorflow implementation of TRPOcode /This is a Tensorflow (v1) implementation of Trust Region Proximal Optimization method. It is purely build on Tensorflow (v1)graphs and encapsulated as a seperate optimizer. You only need to pass the policy function and the cost function to the optimizer and create the cache variables. |
 | RealTimeTrack: A fast algorithm tracking real time deformable planerscode /This is part of my graduation design. It can track multiple deformable images at the same time at a FPS above 30. The idea is based on the paper Template-based Monocular 3D Shape Recovery using Laplacian Meshes but with some performance improvements. |
 | C++ implementation of Selective Search Windowscode /This is a C++ implementation of the paper Segmentation as selective search for object recognition. It reproduced the image segmentation result on Lena. |
| | Semantic 3D Reconstruction from a Collection of Imagescode /This project implemented real-time indoor objects segmentation and 3D reconstruction. We used fine-tuned MaskRCNN doing instance segmentation for 51 different objects and build 3D model by Truncated Signed Distance Function Volume Reconstruction with semantic predicted from MaskRCNN. By now, there are two steps to execute the pipe line. First, download datasets from RGB-D SLAM datasets. Using mask_process.py to generate mask images for specific datasets. Second, change configuration in kernel.cpp to execute TSDF. |
| | Pytorch implementation of Sketch-WGANcode /This is a Pytorch implementation of the paper A Neural Representation of Sketch Drawings. I use WGAN to mimic the data distribution of sketch drawings, where the key difference from the original sketch-rnn is in the reparameterization of GMM. |
| | A tiny renderer based on PBRTcode /This is a naive tiny C++ renderer that is supposed to work on all platforms, based on the famous PBRT book. |
| | PyTorch implementation of QENetcode /This is an unofficial PyTorch implementation of QENet, based on the paper Quaternion Equivariant Capsule Networks for 3D Point Clouds. However, it fails to converge for some unknown reason. I am still invesitigating the issue... |
| | Pytorch Implementation of Various Point Transformerscode /Recently, various methods applied transformers to point clouds: PCT: Point Cloud Transformer (Meng-Hao Guo et al.), Point Transformer (Nico Engel et al.), Point Transformer (Hengshuang Zhao et al.). This repo is a pytorch implementation for these methods and aims to compare them under a fair setting. Currently, all three methods are implemented, while tuning their hyperparameters. |
| | A numpy implementation of PMVScode /This is an unofficial python implementation of PMVS, decribed in Accurate, dense, and robust multi-view stereopsis, PAMI2010. The implementation is slow (no multi-thread and cuda) and it is for only study and illustration purpose. |
| | Perception3D: An Open-Source Deep Learning Framework for 3D Perceptioncode /An Open-Source Deep Learning Framework for 3D Perception (including classification, segmentation, keypoint detection, registration, shape matching, etc.). The documentation (beta) can be built in docs folder. This framework is inspired by mmdetection3d, but with a better configuration system. |
 | Python implementation of PatchMatch Stereocode /This repo (partially) implements patch match stereo algorithm described in PatchMatch Stereo - Stereo Matching with Slanted Support Windows. |
| | Pytorch implementation of OctConvcode /This is a Pytorch implementation of the paper Drop an Octave: Reducing Spatial Redundancy in Convolutional Neural Networks with Octave Convolution |
 | A C++ 3D Object Simplifiercode /This is a C++ implementation of the paper Surface Simplification Using Quadric Error Metrics. It could reduce the 3D model size by merging vertices and faces. |
| | Tensorflow implementation of Multi-stage Reinforcement Learning For Object Detectioncode /This is a Tensorflow implementation of the paper Multi-stage Reinforcement Learning For Object Detection. |
| | A minimal C++ example of Material Point Methodcode /This is a C++ implementation of Material Point Method, with the hybrid Particle-to-Grid and Grid-to-Partical process. |
| | Tiny Mask-RCNN in Tensorflowcode /This is a tiny mask rcnn implemented by Tensorflow with only 700+ lines. Thanks to "tf.map_fn". |
 | Python Mahjong game with monte-carlo AIcode /This is a Mahjong game with GUI. I also implemented a monte-carlo tree search for the AI opponents. |
| | Tensorflow implementation of KdNetcode /This repo is a neat and precise implementation of KdNet (Escape from Cells: Deep Kd-Networks for the Recognition of 3D Point Cloud Models) in Tensorflow with Tensorpack. |
 | C++ implementation of GrabCut algorithmcode /This is a C++ implementation of the famous image segmentation algorithm described in GrabCut: interactive foreground extraction using iterated graph cuts |
| | A geometric processing library in C++ & Pythoncode /This repo implements a simple 3D geometric processing library, including some famous algorithms like geodesic distance computation and Point Pair Features (PPF) computation with CUDA. |
| | Direct Sparse Odometry for iOS platformcode /This is an iOS transporation of Direct Sparse Odometry (https://github.com/JakobEngel/dso). You could press "toggle" to switch between depth images/RGB images/point cloud. |
| | A3C for Doomcode /This is a Tensorflow algorithm to train a RL agent on Doom, with A3C training strategy. |
 | Matlab implementation of Single Image Haze Removal Using Dark Channel Priorcode /This is a Matlab implementation of Kaiming He's famous papaer Single Image Haze Removal Using Dark Channel Prior. |
 | Implementation of Two-Phase Kernel Estimation for Robust Motion Deblurringcode /This is an unofficial python implementation of the deblurring algorithm, decribed in Two-Phase Kernel Estimation for Robust Motion Deblurring, ECCV2010. |
| | Cycle-W-GANcode /This is a Tensorflow implementation of the famous Cycle-GAN described in the paper Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks. I modified it by add Wasserstein distance to make the training more stable. |
| | Tower-Defense Game in Unitycode /This is a tower-defense game written in Unity, a long time ago. It may not be compatible with recent Unity versions. |
| | Pytorch implementation of Learning Latent Subspaces in Variational Autoencoderscode /This is a Pytorch implementation of the paper Learning Latent Subspaces in Variational Autoencoders. It reproduced the experiment on the Swiss Roll toy data. |
| | Tensorflow implementation of Collaborative Learning for Deep Neural Networkscode /This is a Tensorflow implementation of the paper Collaborative Learning for Deep Neural Networks. I got 6.09% error rate after 300 epochs which is a slightly different from the paper. Maybe the split point is different from the paper: in my implementation splitting is done right after Batch Normalization and Relu of transition layers while it is not clear whether they split before or after or in the transition layers. Besides, in my implementation, gradients would pass through soft label targets (notation "q" in the paper). |
 | Pytorch implementation of adversarial autoencoderscode /This repo implements the adversarial autoencoders (https://arxiv.org/pdf/1511.05644.pdf), and reproduced the results on MNIST. One difference is that I used Wasserstein distance instead of naive GAN loss. |