Scheduled maintenance!

We are upgrading our computing nodes to a much higher capacity, that ultimately will increase our service potential. Meanwhile, you may experience longer delays in the execution of your tests.
Please, bear with us while this upgrade takes place.

Please cite us if you find this useful:
C.O.S. Sorzano, J.L. Vilas, E. Ramírez-Aportela, D. del Hoyo, D. Herreros, E. Fernandez-Giménez, D. Marchán, F. de Isidro Gómez, J.R. Macías, I. Sánchez, L. del Caño, Y. Fonseca-Reyna, P. Conesa, A. García-Mena, J. Burguet, J. García Condado, J. Méndez García, M. Martínez, A. Muñoz Barrutia, R. Marabini, J. Vargas, J.M. Carazo.
Image processing tools for the validation of CryoEM maps
Faraday Discuss., 2022,240, 210-227.
Open access (peer-reviewed journals) DOI®: 10.1039/D2FD00059H

(cryo-EM) Validation Report Service

Cryo-electron microscopy is currently one of the most active techniques in Structural Biology. The number of maps deposited at the Electron Microscopy Data Bank is rapidly growing every year and keeping the quality of the submitted maps is essential to maintain the scientific quality of the field.

The ultimate quality measure is the consistency of the map and an atomic model. However, this is only possible for high resolution maps. Over the years there have been many suggestions about validation measures of 3DEM maps. Unfortunately, most of these measures are not currently in use for their spread in multiple software tools and the associated difficulty to access them. To alleviate this problem, we made available through this website a validation grading system that evaluate the information provided to assess the map.

This system grades a map from 0 to 5 depending on the amount of information available. In this way, a map could be validated at Level 0 (the deposited map), 1 (two half maps), 2 (2D classes),
3 (particles), 4 (... + angular assignment), 5 (... + micrographs and coordinates).
In addition, we can have three optional qualifiers: A (... + atomic model), W (... + image processing workflow), and O (... + other techniques).


Getting started

You will be guided through the different steps so you can provide as much or less information (files and parameters) as you may have. Levels are not compulsorily progressive. For instance, you could have validation levels 0 and 2, without having the information for level 1. Although possible, this option is discouraged. You can go back to previous steps when necesary and most of the values entered will be retained, when possible.

Once a job is submitted to the server the execution time varies from 20 minutes to 16 hours, depending on the number of validations to perform, the size of the reconstructed map, and the number of images provided for validation. Also, method A.c is sensible, but takes a lot of time (up to 3 days) to execute due to the molecular dynamics underneath.

When asked for an URL where the image processing workflow can be visualized, you can use Scipion (scipion-em-datamanager) plugin for workflow design. It is available at cryoemworkflowviewer . Although, the validation information ultimately uses the given URL, the workflow will appear in the report only if it was created by Scipion.

The job will be assigned a UUID that you can use to check the progress and retrieve the results upon completion. All provided and temporary data will be securely stored for a limited period of time, and will be automatically removed .

Results will be provided as a PDF report that evaluates the correctness of the submitted map from multiple figures of merit.

References

  • B factor analysis
    Rosenthal, P. B. and Henderson, R. (2003). Optimal determination of particle orientation, absolute hand, and contrast loss in single particle electron-cryomicroscopy. J. Molecular Biology, 333:721–745.
  • DeepRes
    Ramírez-Aportela, E., Mota, J., Conesa, P., Carazo, J. M., and Sorzano, C. O. S. (2019). DeepRes: a new deeplearning- and aspect-based local resolution method for electron-microscopy maps. IUCRj, 6:1054–1063.
  • Local B factor and Local Occupancy
    Kaur, S., Gomez-Blanco, J., Khalifa, A. A., Adinarayanan, S., Sanchez-Garcia, R., Wrapp, D., McLellan, J. S., Bui, K. H., and Vargas, J. (2021). Local computational methods to improve the interpretability and analysis of cryo-EM maps. Nature Communications, 12(1):1–12.
  • DeepHand
    García Condado, J., Muñoz-Burrutia, A., and Sorzano, C. O. S. (2022). Automatic determination of the handedness of single-particle maps of macromolecules solved by CryoEM. J. Structural Biology, 214:107915.

  • Global resolution
    Sorzano, C. O. S., Vargas, J., Oton, J., Abrishami, V., de la Rosa-Trevin, J. M., Gomez-Blanco, J., Vilas, J. L., Marabini, R., and Carazo, J. M. (2017). A review of resolution measures and related aspects in 3D electron microscopy. Progress in biophysics and molecular biology, 124:1–30.
  • FSC Permutation
    Beckers, M. and Sachse, C. (2020). Permutation testing of fourier shell correlation for resolution estimation of cryo-em maps. J. Structural Biology, 212(1):107579.
  • BlocRes
    Cardone, G., Heymann, J. B., and Steven, A. C. (2013). One number does not fit all: Mapping local variations in resolution in cryo-em reconstructions. J. Structural Biology, 184:226–236.
  • ResMap
    Kucukelbir, A., Sigworth, F. J., and Tagare, H. D. (2014). Quantifying the local resolution of cryo-EM density maps. Nature Methods, 11:63–65.
  • MonoRes
    Vilas, J. L., Gómez-Blanco, J., Conesa, P., Melero, R., de la Rosa Trevín, J. M., Otón, J., Cuenca, J., Marabini, R., Carazo, J. M., Vargas, J., and Sorzano, C. O. S. (2018). MonoRes: automatic and unbiased estimation of local resolution for electron microscopy maps. Structure, 26:337–344.
  • MonoDir
    Vilas, J. L., Tagare, H. D., Vargas, J., Carazo, J. M., and Sorzano, C. O. S. (2020). Measuring local-directional resolution and local anisotropy in cryo-EM maps. Nature communications, 11:55.
  • FSO
    Vilas, JL., Tagare, H.D. (2023). New measures of anisotropy of cryo-EM maps. Nature Methods 20, 1021–1024.
  • FSC3D
    Tan, Y., Baldwin, P., Davis, J., Williamson, J., Potter, C., Carragher, B., Lyumkis, D. (2017). Addressing preferred specimen orientation in single-particle cryo-EM through tilting. Nature Methods 14, 793–796.

  • Reprojections Consistency
    Cherian, A., Sra, S., Banerjee, A, Papanikolopoulos, N., (2013). Jensen-Bregman LogDet Divergence with Application to Efficient Similarity Search for Covariance Matrices. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35:2161-2174.

  • Outlier detection
    Sorzano, C. O. S., Vargas, J., de la Rosa-Trevín, J. M., Zaldívar-Peraza, A., Otón, J., Abrishami, V., Foche, I., Marabini, R., Caffarena, G., and Carazo, J. M. (2014). Outlier detection for single particle analysis in electron microscopy. In Proc. Intl. Work-Conference on Bioinformatics and Biomedical Engineering, IWBBIO, page 950.
  • Classification external consistency
    Punjani, A., Brubaker, M. A., and Fleet, D. J. (2017). Building proteins in a day: Efficient 3D molecular structure estimation with electron cryomicroscopy. IEEE Transactions Pattern Analysis & Machine Intelligence, 39:706–718.

  • Similarity Criteria
    Sorzano, C. O. S., Vargas, J., de la Rosa-Trevín, J. M., Otón, J., Alvarez-Cabrera, A. L., Abrishami, V., Sesmero, E., Marabini, R., and Carazo, J. M. (2015). A statistical approach to the initial volume problem in single particle analysis by electron microscopy. J. Structural Biology, 189:213–219.
  • Alignability Smoothness
    Méndez, J., Garduño, E., Carazo, J. M., and Sorzano, C. O. S. (2021). Identification of incorrectly oriented particles in Cryo-EM single particle analysis. J. Structural Biology, 213:107771.
  • Alignability precision and accuracy
    Vargas, J., Otón, J., Marabini, R., Carazo, J. M., and Sorzano, C. O. S. (2016). Particle alignment reliability in single particle electron cryomicroscopy: a general approach. Scientific Reports, 6:21626.

    Vargas, J., Melero, R., Gómez-Blanco, J., Carazo, J. M., and Sorzano, C. O. S. (2017). Quantitative analysis of 3D alignment quality: its impact on soft-validation, particle pruning and homogeneity analysis. Scientific Reports, 7:6307.
  • Relion Alignment and (Relion) Classification without alignment
    Scheres, S. H. W. (2012). A Bayesian view on cryo-EM structure determination. J. Molecular Biology, 415:406–418.
  • CryoSparc Alignment
    Punjani, A., Zhang, H., and Fleet, D. J. (2020). Nonuniform refinement: adaptive regularization improves single-particle cryoEM reconstruction. Nature Methods, 17(12):1214–1221.
  • Overfitting Detection
    Heymann, B. (2015). Validation of 3DEM reconstructions: The phantom in the noise. AIMS Biophysics, 2:21–35.
  • Angular Distribution Efficiency
    Naydenova, K. and Russo, C. J. (2017). Measuring the effects of particle orientation to improve the efficiency of electron cryomicroscopy. Nature communications, 8:629.
  • Sampling Compensation Factor
    Baldwin, P. R. and Lyumkis, D. (2020). Nonuniformity of projection distributions attenuates resolution in Cryo-EM. Progress in Biophysics and Molecular Biology, 150:160–183.

    Baldwin, P. R. and Lyumkis, D. (2021). Tools for visualizing and analyzing Fourier space sampling in Cryo-EM. Progress in Biophysics and Molecular Biology, 160:53-65.
  • CTF Stability
    Fernández-Giménez, E., Carazo, J.M., Sorzano C.O.S..(2023). Local defocus estimation in single particle analysis in cryo-electron microscopy. Journal of Structural Biology, 215(4).

  • Micrograph Cleaner
    Sanchez-Garcia, R., Segura, J., Maluenda, D., Sorzano, C. O. S., and Carazo, J. M. (2020). MicrographCleaner: A python package for cryo-EM micrograph cleaning using deep learning. J. Structural Biology, 210:107498.

  • Phenix validation
    Afonine, P. V., Klaholz, B. P., Moriarty, N. W., Poon, B. K., Sobolev, O. V., Terwilliger, T. C., Adams, P. D., and Urzhumtsev, A. (2018). New tools for the analysis and validation of cryo-EM maps and atomic models. Acta Crystallographica D, Struct. Biol., 74:814–840.
  • FSC-Q
    Ramírez-Aportela, E., Maluenda, D., Fonseca, Y. C., Conesa, P., Marabini, R., Heymann, J. B., Carazo, J. M., and Sorzano, C. O. S. (2021). Fsc-q: A cryoem map-to-atomic model quality validation based on the local fourier shell correlation. Nature Communications, 12(1):1–7.
  • Multimodel
    Herzik, M. A., Fraser, J. S., and Lander, G. C. (2019). A multi-model approach to assessing local and global cryo-EM map quality. Structure, 27(2):344–358.e3.
  • Map-Model Guinier
    Rosenthal, P. B. and Henderson, R. (2003). Optimal determination of particle orientation, absolute hand, and contrast loss in single particle electron-cryomicroscopy. J. Molecular Biology, 333:721–745.
  • MapQ
    Pintilie, G., Zhang, K., Su, Z., Li, S., Schmid, M. F., and Chiu, W. (2020). Measurement of atom resolvability in cryo-em maps with q-scores. Nature methods, 17(3):328–334.
  • EMRinger
    Barad, B. A., Echols, N., Wang, R. Y.-R., Cheng, Y., DiMaio, F., Adams, P. D., and Fraser, J. S. (2015). EMRinger: side chain-directed model and map validation for 3D cryo-electron microscopy. Nature Methods, 12(10):943–946.
  • DAQ
    Terashi, G., Wang, X., Subramaniya, S.R.M.V., Tesmer, J.J.G. and Kihara, D. (2022). Residue-Wise Local Quality Estimation for Protein Models from Cryo-EM Maps. Nature Methods, 19:1116–1125.

  • XLM
    Sinnott, M., Malhotra, S., Madhusudhan, M.~S., Thalassinos, K., and Topf, M. (2020). Combining information from crosslinks and monolinks in the modeling of protein structures. Structure, 28:1061--1070.e3.


  • SAXS
    Jiménez, A., Jonic, S., Majtner, T., Oón, J., Vilas, J. L., Maluenda, D., Mota, J., Ramírez-Aportela, E., Mart´ınez, M., Rancel, Y., Segura, J., Sánchez-García, R., Melero, R., Del Caño, L., Conesa, P., Skjaerven, L., Marabini, R., Carazo, J. M., and Sorzano, C. O. S. (2019). Validation of electron microscopy initial models via small angle X-ray scattering curves. Bioinformatics, 35:2427–2433.
  • Tilt Pair Validation
    Henderson, R., Chen, S., Chen, J. Z., Grigorieff, N., Passmore, L. A., Ciccarelli, L., Rubinstein, J. L., Crowther, R. A., Stewart, P. L., and Rosenthal, P. B. (2011). Tilt-pair analysis of images from a range of different specimens in single-particle electron cryomicroscopy. J. Molecular Biology, 413(5):1028–1046.