Novel View Synthesis for human drawn sketches
Published:
Till now, I researched through a lot of papers and were working with autoencoders to transform the canny edge images from one viewpoint to the other. This approach however doesn’t work on doodle based very lowly abstracted sketches which are more commonly drawn by humans. We tried cyclegan for this approach where we could get the required abstraction to the canny images, however, that approach didn’t give any fruitful results.
Hence, we decided to move to another approach where the 3D model can be retrieved. Hence, I read the following papers :
- Multi-view Supervision for Single-view Reconstruction via Differentiable Ray Consistency
- http://openaccess.thecvf.com/content_cvpr_2018/papers/Tulsiani_Multi-View_Consistency_as_CVPR_2018_paper.pdf
- SketchParse : Towards Rich Descriptions for PoorlyDrawn Sketches using Multi-Task Hierarchical DeepNetworks
- Beyond PASCAL: A Benchmark for 3D Object Detection in the Wild
- etc :
Based on some discussion, we decided to move forward with the approach of pascal 3D+, thus, I explored blender and learnt to deform models. What I got was it was extremely difficult to modify the CAD models, however, it is very easy to make a chair from scratch, but the deformable splines is the major concern that we are interested in.
Thus, we discussed our concern with Prof. Avinash Sharma, he will be sharing some literature regarding that, and then we can figure out our next move.
Adding, to this,I am also hypothesizing that maybe variational auto encoders may give some good results, this approach needs to be discussed in detail though.