| Peer-Reviewed

Quantitative Spatial Analysis on Whole Slide Images Using U-Net

Received: 22 October 2020     Accepted: 18 November 2020     Published: 4 December 2020
Views:       Downloads:
Abstract

Advances in whole slide imaging technology have promoted a high use of digital slide images and generated a large volume of image data that is reliable and useful in determining treatment outcome. Recent technologies closely related to machine learning and deep learning algorithms have contributed to the success of digital histopathology by analyzing the digitized slide images providing quantitative information that are useful for faster turnaround times and effective treatment for the patient. The digital histopathological image analysis has received much attention due to its capability of mitigating the problem of the hand-crafted features. Features directly learned from raw data are trainable within the deep learning procedure and can be used for the histopathology image classification task. However, understanding the spatial context of cancer cells is still a challenging issue because of the heterogeneity of the tumor microenvironment which varies greatly, preventing successful diagnosis and leads to inappropriate therapeutic approaches for cancer patients. In this paper, we present a spatial analysis method for tumor microenvironment analysis using the U-Net architecture, a semantic segmentation deep-learning model, for a better understanding of the spatial relations between tissue types. We demonstrate the effectiveness of the U-Net architecture using a dataset created by an international crowdsourcing study. Moreover, we show that the quantitative estimates can be derived from the univariate spatial analysis.

Published in Computational Biology and Bioinformatics (Volume 8, Issue 2)
DOI 10.11648/j.cbb.20200802.18
Page(s) 90-96
Creative Commons

This is an Open Access article, distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution and reproduction in any medium or format, provided the original work is properly cited.

Copyright

Copyright © The Author(s), 2020. Published by Science Publishing Group

Keywords

Cancer, Whole Slide Images, Spatial Analysis, U-Net, Machine Learning

References
[1] Pantanowitz, L., Sinard, J. H., Henricks, W. H., Fatheree, L. A., Carter, A. B., Contis, L., Beckwith, B. A., Evans, A. J., Lal, A. and Parwani, A. V., 2013. Validating whole slide imaging for diagnostic purposes in pathology: guideline from the College of American Pathologists Pathology and Laboratory Quality Center. Archives of Pathology and Laboratory Medicine, 137 (12), pp. 1710-1722.
[2] Abels, E. and Pantanowitz, L., 2017. Current state of the regulatory trajectory for whole slide imaging devices in the USA. Journal of pathology informatics, 8.
[3] Komura, D. and Ishikawa, S., 2018. Machine learning methods for histopathological image analysis. Computational and structural biotechnology journal, 16, pp. 34-42.
[4] Lee, S., Amgad, M., Chittajallu, D. R., McCormick, M., Pollack, B. P., Elfandy, H., Hussein, H., Gutman, D. A. and Cooper, L. A., 2020. HistomicsML2. 0: Fast interactive machine learning for whole slide imaging data. arXiv preprint arXiv: 2001.11547.
[5] Nalisnik, M., Amgad, M., Lee, S., Halani, S. H., Vega, J. E. V., Brat, D. J., Gutman, D. A. and Cooper, L. A., 2017. Interactive phenotyping of large-scale histology imaging data with HistomicsML. Scientific reports, 7 (1), pp. 1-12.
[6] Lee, S., Amgad, M., Masoud, M., Subramanian, R., Gutman, D. and Cooper, L., 2019, November. An Ensemble-based Active Learning for Breast Cancer Classification. In 2019 IEEE International Conference on Bioinformatics and Biomedicine (BIBM) (pp. 2549-2553). IEEE.
[7] Cruz-Roa, A., Gilmore, H., Basavanhally, A., Feldman, M., Ganesan, S., Shih, N. N., Tomaszewski, J., González, F. A. and Madabhushi, A., 2017. Accurate and reproducible invasive breast cancer detection in whole-slide images: A Deep Learning approach for quantifying tumor extent. Scientific reports, 7, p. 46450.
[8] LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015 May 28; 521 (7553): 436-44. doi: 10.1038/nature14539. PMID: 26017442.
[9] Litjens G, Sánchez CI, Timofeeva N, Hermsen M, Nagtegaal I, Kovacs I, Hulsbergen-van de Kaa C, Bult P, van Ginneken B, van der Laak J. Deep learning as a tool for increased accuracy and efficiency of histopathological diagnosis. Sci Rep. 2016 May 23; 6: 26286. doi: 10.1038/srep26286. PMID: 27212078; PMCID: PMC4876324.
[10] Hou L, Samaras D, Kurc TM, Gao Y, Davis JE, Saltz JH. Patch-based Convolutional Neural Network for Whole Slide Tissue Image Classification. Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit. 2016 Jun-Jul; 2016: 2424-2433. doi: 10.1109/CVPR.2016.266. PMID: 27795661; PMCID: PMC5085270.
[11] Sun W, Zheng B, Qian W. Automatic feature learning using multichannel ROI based on deep structured algorithms for computerized lung cancer diagnosis. Comput Biol Med. 2017 Oct 1; 89: 530-539. doi: 10.1016/j.compbiomed.2017.04.006. Epub 2017 Apr 13. PMID: 28473055.
[12] Murthy V, Hou L, Samaras D, Kurc TM, Saltz JH. Center-Focusing Multi-task CNN with Injected Features for Classification of Glioma Nuclear Images. IEEE Winter Conf Appl Comput Vis. 2017 Mar; 2017: 834-841. doi: 10.1109/WACV.2017.98. Epub 2017 May 15. PMID: 29881826; PMCID: PMC5988234.
[13] Xu Y, Jia Z, Wang LB, Ai Y, Zhang F, Lai M, Chang EI. Large scale tissue histopathology image classification, segmentation, and visualization via deep convolutional activation features. BMC Bioinformatics. 2017 May 26; 18 (1): 281. doi: 10.1186/s12859-017-1685-x. PMID: 28549410; PMCID: PMC5446756.
[14] Campanella, G., Hanna, M. G., Geneslaw, L., Miraflor, A., Silva, V. W. K., Busam, K. J., Brogi, E., Reuter, V. E., Klimstra, D. S. and Fuchs, T. J., 2019. Clinical-grade computational pathology using weakly supervised deep learning on whole slide images. Nature medicine, 25 (8), pp. 1301-1309.
[15] Kothari, S., Phan, J. H., Stokes, T. H. and Wang, M. D., 2013. Pathology imaging informatics for quantitative analysis of whole-slide images. Journal of the American Medical Informatics Association, 20 (6), pp. 1099-1108.
[16] Topol, E J., 2019. High-performance medicine: the convergence of human and artificial intelligence. Nature medicine, 25 (1), pp. 44-56.
[17] Vandenberghe, M. E., Scott, M. L., Scorer, P. W., Söderberg, M., Balcerzak, D. and Barker, C., 2017. Relevance of deep learning to facilitate the diagnosis of HER2 status in breast cancer. Scientific reports, 7, p. 45938.
[18] Joyce JA. Therapeutic targeting of the tumor microenvironment. Cancer cell. 7 (6): 513-20, 2005.
[19] Lorusso G, Ruegg C. The tumor microenvironment and its contribution to tumor evolution toward metastasis. Histochemistry and cell biology. 130 (6): 1091-103, 2008.
[20] Yu KH, Zhang C, Berry GJ, Altman RB, Ré C, Rubin DL, Snyder M. Predicting non-small cell lung cancer prognosis by fully automated microscopic pathology image features. Nat Commun. 2016 Aug 16; 7: 12474. doi: 10.1038/ncomms12474. PMID: 27527408; PMCID: PMC4990706.
[21] Ehteshami Bejnordi B, Balkenhol M, Litjens G, Holland R, Bult P, Karssemeijer N, van der Laak JA. Automated Detection of DCIS in Whole-Slide H&E Stained Breast Histopathology Images. IEEE Trans Med Imaging. 2016 Sep; 35 (9): 2141-2150. doi: 10.1109/TMI.2016.2550620. Epub 2016 Apr 5. PMID: 27076354.
[22] Tomczak, K., Czerwińska, P., and Wiznerowicz, M.: ‘The Cancer Genome Atlas (TCGA): an immeasurable source of knowledge’, Contemp Oncol (Pozn), 2015, 19, (1A), pp. A68-A77.
[23] Amgad, M., Elfandy, H., Hussein, H., Atteya, L. A., Elsebaie, M. A., Abo Elnasr, L. S., Sakr, R. A., Salem, H. S., Ismail, A. F., Saad, A. M. and Ahmed, J., 2019. Structured crowdsourcing enables convolutional segmentation of histology images. Bioinformatics, 35 (18), pp. 3461-3467.
[24] Long, J., Shelhamer, E. and Darrell, T., 2015. Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 3431-3440).
[25] Ronneberger, O., Fischer, P. and Brox, T., 2015, October. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention (pp. 234-241). Springer, Cham.
[26] Jana M, Sar N. Modeling of hotspot detection using cluster outlier analysis and Getis-Ord Gi* statistic of educational development in upper-primary level, India. Modeling Earth Systems and Environment. 2 (2): 60, 2016.
[27] Junior GB, Paiva AC, Silva AC, de Oliveira AC. Classification of breast tissues using Getis-Ord statistics and support vector machines. Intelligent Decision Technologies. 3 (4): 197-205, 2009.
Cite This Article
  • APA Style

    Sanghoon Lee, Yanjun Zhao, Mohamed Masoud, Saeid Belkasim. (2020). Quantitative Spatial Analysis on Whole Slide Images Using U-Net. Computational Biology and Bioinformatics, 8(2), 90-96. https://doi.org/10.11648/j.cbb.20200802.18

    Copy | Download

    ACS Style

    Sanghoon Lee; Yanjun Zhao; Mohamed Masoud; Saeid Belkasim. Quantitative Spatial Analysis on Whole Slide Images Using U-Net. Comput. Biol. Bioinform. 2020, 8(2), 90-96. doi: 10.11648/j.cbb.20200802.18

    Copy | Download

    AMA Style

    Sanghoon Lee, Yanjun Zhao, Mohamed Masoud, Saeid Belkasim. Quantitative Spatial Analysis on Whole Slide Images Using U-Net. Comput Biol Bioinform. 2020;8(2):90-96. doi: 10.11648/j.cbb.20200802.18

    Copy | Download

  • @article{10.11648/j.cbb.20200802.18,
      author = {Sanghoon Lee and Yanjun Zhao and Mohamed Masoud and Saeid Belkasim},
      title = {Quantitative Spatial Analysis on Whole Slide Images Using U-Net},
      journal = {Computational Biology and Bioinformatics},
      volume = {8},
      number = {2},
      pages = {90-96},
      doi = {10.11648/j.cbb.20200802.18},
      url = {https://doi.org/10.11648/j.cbb.20200802.18},
      eprint = {https://article.sciencepublishinggroup.com/pdf/10.11648.j.cbb.20200802.18},
      abstract = {Advances in whole slide imaging technology have promoted a high use of digital slide images and generated a large volume of image data that is reliable and useful in determining treatment outcome. Recent technologies closely related to machine learning and deep learning algorithms have contributed to the success of digital histopathology by analyzing the digitized slide images providing quantitative information that are useful for faster turnaround times and effective treatment for the patient. The digital histopathological image analysis has received much attention due to its capability of mitigating the problem of the hand-crafted features. Features directly learned from raw data are trainable within the deep learning procedure and can be used for the histopathology image classification task. However, understanding the spatial context of cancer cells is still a challenging issue because of the heterogeneity of the tumor microenvironment which varies greatly, preventing successful diagnosis and leads to inappropriate therapeutic approaches for cancer patients. In this paper, we present a spatial analysis method for tumor microenvironment analysis using the U-Net architecture, a semantic segmentation deep-learning model, for a better understanding of the spatial relations between tissue types. We demonstrate the effectiveness of the U-Net architecture using a dataset created by an international crowdsourcing study. Moreover, we show that the quantitative estimates can be derived from the univariate spatial analysis.},
     year = {2020}
    }
    

    Copy | Download

  • TY  - JOUR
    T1  - Quantitative Spatial Analysis on Whole Slide Images Using U-Net
    AU  - Sanghoon Lee
    AU  - Yanjun Zhao
    AU  - Mohamed Masoud
    AU  - Saeid Belkasim
    Y1  - 2020/12/04
    PY  - 2020
    N1  - https://doi.org/10.11648/j.cbb.20200802.18
    DO  - 10.11648/j.cbb.20200802.18
    T2  - Computational Biology and Bioinformatics
    JF  - Computational Biology and Bioinformatics
    JO  - Computational Biology and Bioinformatics
    SP  - 90
    EP  - 96
    PB  - Science Publishing Group
    SN  - 2330-8281
    UR  - https://doi.org/10.11648/j.cbb.20200802.18
    AB  - Advances in whole slide imaging technology have promoted a high use of digital slide images and generated a large volume of image data that is reliable and useful in determining treatment outcome. Recent technologies closely related to machine learning and deep learning algorithms have contributed to the success of digital histopathology by analyzing the digitized slide images providing quantitative information that are useful for faster turnaround times and effective treatment for the patient. The digital histopathological image analysis has received much attention due to its capability of mitigating the problem of the hand-crafted features. Features directly learned from raw data are trainable within the deep learning procedure and can be used for the histopathology image classification task. However, understanding the spatial context of cancer cells is still a challenging issue because of the heterogeneity of the tumor microenvironment which varies greatly, preventing successful diagnosis and leads to inappropriate therapeutic approaches for cancer patients. In this paper, we present a spatial analysis method for tumor microenvironment analysis using the U-Net architecture, a semantic segmentation deep-learning model, for a better understanding of the spatial relations between tissue types. We demonstrate the effectiveness of the U-Net architecture using a dataset created by an international crowdsourcing study. Moreover, we show that the quantitative estimates can be derived from the univariate spatial analysis.
    VL  - 8
    IS  - 2
    ER  - 

    Copy | Download

Author Information
  • Department of Computer Sciences and Electrical Engineering, Marshall University, Huntington, United States of America

  • Department of Computer Science, Troy University, Troy, United States of America

  • Department of Neurology, Emory University, Atlanta, United States of America

  • Department of Computer Science, Georgia State University, Atlanta, United States of America

  • Sections