综合对比：
| paper | code | Method | KITTI 2012 | KITTI 2015 | Sintel Clean | Sintel Final |
|---|---|---|---|---|---|---|
| train | test | train | test(F1-all) | |||
| Jason J Y, Harley A W, Derpanis K G. Back to basics: Unsupervised learning of optical flow via brightness constancy and motion smoothness[C]//European Conference on Computer Vision. Springer, Cham, 2016: 3-10. | https://github.com/ryersonvisionlab/unsupFlownet | BackToBasic | 11.3 | 9.9 | – | – |
| Ren Z, Yan J, Ni B, et al. Unsupervised deep learning for optical flow estimation[C]//Thirty-First AAAI Conference on Artificial Intelligence. 2017. | https://github.com/sunshinezhe/Dense-Spatial-Transform-Flow | DSTFlow | 10.43 | 12.4 | 16.79 | 39% |
| Meister S, Hur J, Roth S. Unflow: Unsupervised learning of optical flow with a bidirectional census loss[C]//Thirty-Second AAAI Conference on Artificial Intelligence. 2018. | https://github.com/simonmeister/UnFlow | UnFlow | 3.29 | – | 8.1 | 23.30% |
| Wang Y, Yang Y, Yang Z, et al. Occlusion aware unsupervised learning of optical flow[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018: 4884-4893. | – | OAFlow | 3.55 | 4.2 | 8.88 | 31.20% |
| Janai J, Guney F, Ranjan A, et al. Unsupervised learning of multi-frame optical flow with occlusions[C]//Proceedings of the European Conference on Computer Vision (ECCV). 2018: 690-706. | https://github.com/JJanai/back2future | Back2Future | – | – | 6.59 | 22.94% |
| Tian L, Tu Z, Zhang D, et al. Unsupervised learning of optical flow with cnn-based non-local filtering[J]. IEEE Transactions on Image Processing, 2020, 29: 8429-8442. | – | NLFlow | 3.02 | 4.5 | 6.05 | 22.75% |
| Liu P, King I, Lyu M R, et al. Ddflow: Learning optical flow with unlabeled data distillation[C]//Proceedings of the AAAI Conference on Artificial Intelligence. 2019, 33(01): 8770-8777. | https://github.com/ppliuboy/DDFlow | DDFlow | 2.35 | 3 | 5.72 | 14.29% |
| Zhong Y, Ji P, Wang J, et al. Unsupervised deep epipolar flow for stationary or dynamic scenes[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019: 12095-12104. | https://github.com/yiranzhong/EPIflow （loss functions地址） | EpiFlow | 2.51 | 3.4 | 5.55 | 16.95% |
| Liu P, Lyu M, King I, et al. Selflow: Self-supervised learning of optical flow[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019: 4571-4580. | https://github.com/ppliuboy/SelFlow | SelFlow | 1.69 | 2.2 | 4.84 | 14.19% |
| Tian L, Tu Z, Zhang D, et al. Unsupervised learning of optical flow with cnn-based non-local filtering[J]. IEEE Transactions on Image Processing, 2020, 29: 8429-8442. | – | STFlow | 1.64 | 1.9 | 3.56 | 13.83% |
| Liu L, Zhang J, He R, et al. Learning by analogy: Reliable supervision from transformations for unsupervised optical flow estimation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020: 6489-6498. | https://github.com/lliuz/ARFlow | ARFlow | 1.44 | 1.8 | 2.85 | 11.80% |
| Im W, Kim T K, Yoon S E. Unsupervised learning of optical flow with deep feature similarity[C]//European Conference on Computer Vision. Springer, Cham, 2020: 172-188. | https://github.com/iwbn/unsupsimflow | SimFlow | – | – | 5.19 | 13.38% |
| Jonschkowski R, Stone A, Barron J T, et al. What matters in unsupervised optical flow[C]//Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part II 16. Springer International Publishing, 2020: 557-572. | https://github.com/google-research/google-research/tree/master/uflow | UFlow | 1.68 | 1.9 | 2.71 | 11.13% |
| Luo K, Wang C, Liu S, et al. Upflow: Upsampling pyramid for unsupervised optical flow learning[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021: 1045-1054. | https://github.com/coolbeam/UPFlow_pytorch | UPFlow | 1.27 | 1.4 | 2.45 | 9.38% |
| Luo K, Luo A, Wang C, et al. ASFlow: Unsupervised Optical Flow Learning with Adaptive Pyramid Sampling[J]. arXiv preprint arXiv:2104.03560, 2021. | – | ASFlow | 1.26 | 1.5 | 2.47 | 9.67% |
| Li J, Zhao J, Song S, et al. Occlusion aware unsupervised learning of optical flow from video[C]//Thirteenth International Conference on Machine Vision. International Society for Optics and Photonics, 2021, 11605: 116050T. | https://github.com/CV-IP/UnOpticalFlow | UnOpticalFlow | 2.67 | 7.1 | 22% | – |
| Luo K, Wang C, Ye N, et al. Occinpflow: Occlusion-inpainting optical flow estimation by unsupervised learning[J]. arXiv preprint arXiv:2006.16637, 2020. | https://github.com/coolbeam/OccInpFlow#occinpflow-occlusion-inpainting-optical-flow-estimation-by-unsupervised-learning | OccInpFlow | 1.78 | 2.1 | 4.57 | 15.20% |
-
研究建议一 ：可以与《Neural-network-based Motion Tracking for Breast Ultrasound Strain Elastography: An Initial Assessment of erformance and Feasibility》文章类似思路。研究并比较上述自监督模型在超声B模式图像上在心动图上的一个数据情况。同时可以使用8个模拟心脏的模型和有限元和超声模拟数据进行验证。训练数据可以直接采用超声B模式的图像。
-
*研究建议二：如果上述具有可行性的话，因为里面如ARFlow模型借鉴了PWC-Net模型的金字塔网络的输入部分。那么就同理可以借鉴RFPWC-Net模型的输入部分。然后改损失函数。实现利用RF数据来作为训练模型的输入，原来使用RF数据训练模型最大的问题就是label不好确定。但是如果采用自监督模型的话就解决了最大的问题。然后两个结合起来应该是效果不错的，但是工作量比较大。后续可以挖掘的空间还很大。是一个很不错的点。