Data Management Policy
Read the DOE Office of Science Statement on Digital Data Management:
http://science.energy.gov/funding-opportunities/digital-data-management/
In keeping with its role as a federally-funded User Facility, it is the DOE JGI’s explicit policy to make the data it produces during the conduct of user projects (primarily but not exclusively its Community Sequencing Program, CSP) available to the broad scientific community as rapidly as possible and in accordance with DOE Office of Science policies. The DOE JGI serves raw and processed data to users through its individual genome project data portals (accessed through the Project List, http://genome.jgi.doe.gov/) as well as its several comparative genomics platforms (Phytozome, MycoCosm and IMG). The DOE JGI provides permanent local archiving of these data in addition to depositing raw data, assemblies and annotations into archives maintained by the National Center for Biotechnology Information (NCBI). Users of DOE JGI data must abide by the DOE JGI’s data release policy and accepted community standards that reserve the right to first publication of genome analyses for the data producers as outlined in the Fort Lauderdale (http://www.sanger.ac.uk/datasharing/assets/fortlauderdalereport.pdf) and Toronto (Nature 461:168-170, 2009) agreements.
It is expected that additional data and analyses produced externally by users in support of the DOE JGI user projects will be made accessible to the public at the time of their publication. Archiving and maintaining public access to user-created data and analyses is the responsibility of the project Principal Investigator unless otherwise agreed to by the DOE JGI. Such arrangements should be specified in the User Agreement.
Current Data Management Practices
Raw sequence data—Raw data produced by the DOE JGI’s sequencing platforms are base-called in real time and primary (image) data are discarded shortly after base-calling is complete. Sequence data are stored as FASTQ files containing the linear sequence and associated quality scores. All primary sequence data receive automated quality control screening to assess sequence quality and quantity, identify major contamination, and confirm that the data generated correspond to the expected organism/environment being sequenced. For most sequencing projects, raw sequence data is made available to registered users through the genome-specific portal web page and are submitted to NCBI’s Sequence Read Archive (SRA) as soon as practicable, generally within 30 days.
Processed sequence data—For most user projects, the DOE JGI conducts standard post-sequencing analyses that vary by project type (see Product Offerings). These analyses and associated metadata are available for download from project-specific data portals to the DOE JGI data users. User-selected subsets of public data can also be downloaded from the DOE JGI’s comparative analysis platforms as described in Data Resources below. Beginning in fall 2013 users wishing to download data from the DOE JGI portals will be required to register with the DOE JGI.
Project metadata—Every user project has associated metadata that are important for interpretation of the data produced. These include, for example, authors of the data, PI contact information, detailed sample information and details of analyses performed. These data will be served with processed data from the DOE JGI’s data portals whenever possible, however metadata for older legacy projects may be limited.
Data archive and back-up—All data available for download by users is automatically archived onto a high-speed tape system located at the National Energy Research Scientific Computing facility (NERSC). A second redundant copy of these data is also stored on high-speed tape (but not on the same cassette!). Typical retrieval times range from minutes to hours depending primarily on file size.
Data Management Resources
The DOE JGI provides some resources to users to facilitate analysis of genomic data produced outside of the DOE JGI or to allow post-hoc analysis of DOE JGI data. Because these resources are supported through public funds it is expected that such analyses and externally created data managed with these resources will be made public at the time of publication. The user bears sole responsibility for providing and maintaining public access to these data and analyses.
Phytozome —The DOEJGI’s comparative genomics platform for plants allows user-driven filtering of plant genome data by characteristics of interest that are then downloadable by individual users. Phytozome does not currently keep these analyses nor support their display for other users. Phytozome supports the display of user-provided genome tracks that are compatible with GBrowse, but does not generally display these tracks for other users. By mutual agreement between JGI and the data providers, Phytozome may display non-DOE JGI genomes for comparative purposes or additional genome-anchored data for DOE JGI genomes. The DOE JGI will make these data immediately public and downloadable, but responsibility for depositing these data in public repositories resides with the data provider.
MycoCosm —The DOEJGI’s comparative genomics platform for fungi supports user-driven filtering of comparative genomics data by characteristics of interest that are downloadable by individual users. MycoCosm supports manual curation of genome annotations by registered users who have received specific training. These annotations are in turn served to the user community. MycoCosm may display genomes and annotations created elsewhere by mutual agreement between JGI and the owners of the external genome data. JGI will make these data immediately public and downloadable, but responsibility for depositing these data in public repositories resides with the data provider.
Integrated Microbial Genomes platform (IMG)—IMG supports user-driven filtering of comparative genomics data by characteristics of interest and maintains results for users. Results of comparative analyses are downloadable and may be shared with other users within the IMG system. Responsibility for providing public access to these data at the time of publication is the sole responsibility of the user. Isolate genomes, single cell genomes and metagenome datasets produced elsewhere may be uploaded for annotation and analysis within IMG. Data must be made public at the time of publication and this responsibility lies with the user.
Genomes OnLine Database (GOLD)—GOLD is an online database of worldwide genome projects and associated metadata that is maintained for the public by the DOE JGI. The data are filterable and downloadable. Users may register their genome projects in GOLD, but must agree to make the registration and associated metadata publicly available at the time of publication. Registration of non-DOE JGI genome projects with NCBI or other public registries is the sole responsibility of the user.
DOE JGI’s Policy on Microbial Genbank Submissions
Microbial genome sequencing projects become eligible for preparation for submission to Genbank when they become public in the Integrated Microbial Genomes system, according to the DOE embargo policy for release of genome data. This includes sequences from both cultured organisms and single cells* sequenced at DOE JGI only**.
All eligible projects are prepared as WGS submissions. Although JGI has been submitting annotated genomes to Genbank in the past, moving forward we will only support WGS submissions and will serve annotations through IMG.
Microbial submissions to Genbank may be tracked at: https://mgs.jgi-psf.org (requires login). A JGI single sign-on account is required to view submission status.
For questions, please contact Galina Ovchinnikova.
* Beginning in April 2014, all single cells sequenced at the JGI will go through JGI’s single-cell contamination screening before being submitted to IMG or being prepared for submission to Genbank.
** Please note that genomes submitted by external users for annotation and comparative analysis in IMG are not automatically submitted to Genbank.