DOE Joint Genome Institute

  • COVID-19
  • About Us
  • Contact Us
  • Our Science
    • DOE Mission Areas
    • Bioenergy Research Centers
    • Science Programs
    • Science Highlights
    • Scientists
    (PXFuel)
    Designer DNA: JGI Helps Users Blaze New Biosynthetic Pathways
    In a special issue of the journal Synthetic Biology, JGI scientific users share how they’ve worked with the JGI DNA Synthesis Science Program and what they’ve discovered through their collaborations.

    More

    A genetic element that generates targeted mutations, called diversity-generating retroelements (DGRs), are found in viruses, as well as bacteria and archaea. Most DGRs found in viruses appear to be in their tail fibers. These tail fibers – signified in the cartoon by the blue virus’ downward pointing ‘arms’— allow the virus to attach to one cell type (red), but not the other (purple). DGRs mutate these ‘arms,’ giving the virus opportunities to switch to different prey, like the purple cell. (Courtesy of Blair Paul)
    A Natural Mechanism Can Turbocharge Viral Evolution
    A team has discovered that diversity generating retroelements (DGRs) are not only widespread, but also surprisingly active. In viruses, DGRs appear to generate diversity quickly, allowing these viruses to target new microbial prey.

    More

    Algae growing in a bioreactor. (Dennis Schroeder, NREL)
    Refining the Process of Identifying Algae Biotechnology Candidates
    Researchers combined expertise at the National Labs to screen, characterize, sequence and then analyze the genomes and multi-omics datasets for algae that can be used for large-scale production of biofuels and bioproducts.

    More

  • Our Projects
    • Search JGI Projects
    • DOE Metrics/Statistics
    • Approved User Proposals
    • Legacy Projects
    This data image shows the monthly average sea surface temperature for May 2015. Between 2013 and 2016, a large mass of unusually warm ocean water--nicknamed the blob--dominated the North Pacific, indicated here by red, pink, and yellow colors signifying temperatures as much as three degrees Celsius (five degrees Fahrenheit) higher than average. Data are from the NASA Multi-scale Ultra-high Resolution Sea Surface Temperature (MUR SST) Analysis product. (Courtesy NASA Physical Oceanography Distributed Active Archive Center)
    When “The Blob” Made It Hotter Under the Water
    Researchers tracked the impact of a large-scale heatwave event in the ocean known as “The Blob” as part of an approved proposal through the Community Science Program.

    More

    A plantation of poplar trees. (David Gilbert)
    Genome Insider podcast: THE Bioenergy Tree
    The US Department of Energy’s favorite tree is poplar. In this episode, hear from ORNL scientists who have uncovered remarkable genetic secrets that bring us closer to making poplar an economical and sustainable source of energy and materials.

    More

    Ian Rambo, graduate student at UT-Austin, was a DOE Graduate Student Research Fellow at the JGI
    Virus-Microbe Interactions of Mud Island Mangroves
    Through the DOE Office of Science Graduate Student Research (SCGSR) program, Ian Rambo worked on part of his dissertation at the JGI. The chapter focuses on how viruses influence carbon cycling in coastal mangroves.

    More

  • Data & Tools
    • IMG
    • Data Portal
    • MycoCosm
    • PhycoCosm
    • Phytozome
    • GOLD
    HPCwire Editor's Choice Award (logo crop) for Best Use of HPC in the Life Sciences
    JGI Part of Berkeley Lab Team Awarded Best Use of HPC in Life Sciences
    The HPCwire Editors Choice Award for Best Use of HPC in Life Sciences went to the Berkeley Lab team comprised of JGI and ExaBiome Project team, supported by the DOE Exascale Computing Project for MetaHipMer, an end-to-end genome assembler that supports “an unprecedented assembly of environmental microbiomes.”

    More

    With a common set of "baseline metadata," JGI users can more easily access public data sets. (Steve Wilson)
    A User-Centered Approach to Accessing JGI Data
    Reflecting a structural shift in data access, the JGI Data Portal offers a way for users to more easily access public data sets through a common set of metadata.

    More

    Phytozome portal collage
    A More Intuitive Phytozome Interface
    Phytozome v13 now hosts upwards of 250 plant genomes and provides users with the genome browsers, gene pages, search, BLAST and BioMart data warehouse interfaces they have come to rely on, with a more intuitive interface.

    More

  • User Programs
    • Calls for Proposals
    • Special Initiatives & Programs
    • Product Offerings
    • User Support
    • Policies
    • Submit a Proposal
    screencap from Amundson and Wilkins subsurface microbiome video
    Digging into Microbial Ecosystems Deep Underground
    JGI users and microbiome researchers at Colorado State University have many questions about the microbial communities deep underground, including the role viral infection may play in other natural ecosystems.

    Read more

    Yeast strains engineered for the biochemical conversion of glucose to value-added products are limited in chemical output due to growth and viability constraints. Cell extracts provide an alternative format for chemical synthesis in the absence of cell growth by isolating the soluble components of lysed cells. By separating the production of enzymes (during growth) and the biochemical production process (in cell-free reactions), this framework enables biosynthesis of diverse chemical products at volumetric productivities greater than the source strains. (Blake Rasor)
    Boosting Small Molecule Production in Super “Soup”
    Researchers supported through the Emerging Technologies Opportunity Program describe a two-pronged approach that starts with engineered yeast cells but then moves out of the cell structure into a cell-free system.

    More

    These bright green spots are fluorescently labelled bacteria from soil collected from the surface of plant roots. For reference, the scale bar at bottom right is 10 micrometers long. (Rhona Stuart)
    A Powerful Technique to Study Microbes, Now Easier
    In JGI's Genome Insider podcast: LLNL biologist Jennifer Pett-Ridge collaborated with JGI scientists through the Emerging Technologies Opportunity Program to semi-automate experiments that measure microbial activity in soil.

    More

  • News & Publications
    • News
    • Blog
    • Podcasts
    • Webinars
    • Publications
    • Newsletter
    • Logos and Templates
    • Photos
    In their approved proposal, Frederick Colwell of Oregon State University and colleagues are interested in the microbial communities that live on Alaska’s glacially dominated Copper River Delta. They’re looking at how the microbes in these high latitude wetlands, such as the Copper River Delta wetland pond shown here, cycle carbon. (Courtesy of Rick Colwell)
    Monitoring Inter-Organism Interactions Within Ecosystems
    Many of the proposals approved through JGI's annual Community Science Program call focus on harnessing genomics to developing sustainable resources for biofuels and bioproducts.

    More

    Coloring the water, the algae Phaeocystis blooms off the side of the sampling vessel, Polarstern, in the temperate region of the North Atlantic. (Katrin Schmidt)
    Climate Change Threatens Base of Polar Oceans’ Bountiful Food Webs
    As warm-adapted microbes edge polewards, they’d oust resident tiny algae. It's a trend that threatens to destabilize the delicate marine food web and change the oceans as we know them.

    More

    Integrating JGI Capabilities for Exploring Earth’s Secondary Metabolome
    Natural Prodcast podcast: Nigel Mouncey
    JGI Director Nigel Mouncey has a vision to build out an integrative genomics approach to looking at the interactions of organisms and environments. He also sees secondary metabolism analysis and research as a driver for novel technologies that can serve all JGI users.

    More

Data & Tools
Home › Data & Tools › Software › BBTools › BBTools User Guide › BBMask Guide

BBMask Guide

BBMask is designed for masking sequence, primarily to prevent false-positive matches in highly-conserved or low-complexity regions of genomes. It was designed as a replacement for tools like Dust which are incredibly slow and do not work well for that purpose. It does three types of masking, all optional:
1) Low-entropy (complexity)
2) Tandem-repeated kmers
3) Sam-file coverage

*Notes*

Entropy:

Entropy is calculated using Shannon Entropy of kmers in a window, and varies from 0 (mask nothing) to 1 (mask everything). It’s a little hard to figure out exactly what threshold you should set, but for reference, the default of 0.7 masks only 107 bases in the E.coli genome and about 0.7% of the human genome. 0.9 masks about 8kbp of E.coli and 7% of the human genome.

Memory:

BBMask loads all sequences into memory to allow multiple masking operations. For that reason, it requires about 1 byte per base (for fasta).

Human, Cat, Dog, and Mouse removal:

Masked references of various vertebrate organisms were prepared using BBMask. These are used by JGI to remove contaminant reads from libraries with zero false-positives. To produce them, BBMask was used, with default settings. Additionally, all of Mycocosm and Phytozome were shredded into 80bp overlapping pieces and mapped to the genomes; the resulting sam files were used for masking. Some organisms such as Maize were not used due to a high level of human contamination; this was ascertained by manually looking for long regions in the mapped fungal and plant genomes that had mapped to human at very high identity (typically 100% over a few hundred bp). These were BLASTed; if they hit other primates but no other plants or fungus, they were determined to be human contamination in the other organism’s reference, and thus NOT masked. If they hit other related plants and fungi but no animals other than human, they were determined to be plant/fungal contamination in the human reference, and masked. Due to the relatively high level of apparent contamination in nonprimary human reference sequences (such as unplaced and alternate contigs), none of these are used. It is also possible to mask ribosomal sequences as well, since they are highly conserved. This was done in the case of human, using Silva.

A handful of animals were also used for masking, including zebra danio (the only vertebrate used). The goal here was to bottleneck homologous regions. This is a highly simplified explanation, but my justification is this: If a gene in human is shared, largely unchanged, with a fungus, that means it had to go through fish, since fish are evolutionarily between humans and fungi. Therefore, masking human with fish should remove any sequences that are shared largely unchanged between humans and fungi. It’s a safe operation since the goal here is to achieve zero false positive read removals.

*Usage Examples*

To mask using just entropy:

bbmask.sh in=ref.fa out=masked.fa entropy=0.7

To mask sequences in genome A similar to those in genome B, plus low-entropy sequences:
shred.sh in=B.fa out=shredded.fa length=80 minlength=70 overlap=40
bbmap.sh ref=A.fa in=shredded.fa outm=mapped.sam minid=0.85 maxindel=2
bbmask.sh in=A.fa out=masked.fa entropy=0.7 sam=mapped.sam

To filter low-entropy sequences rather than masking them:
See the BBDuk Guide.

  • BBTools User Guide
    • Installation Guide
    • Usage Guide
    • Data Preprocessing
    • Add Adapters Guide
    • BBDuk Guide
    • BBMap Guide
    • BBMask Guide
    • BBMerge Guide
    • BBNorm Guide
    • CalcUniqueness Guide
    • Clumpify Guide
    • Dedupe Guide
    • Reformat Guide
    • Repair Guide
    • Seal Guide
    • Split Nextera Guide
    • Statistics Guide
    • Tadpole Guide
    • Taxonomy Guide
  • BBTools FAQ and Support Forums

More topics:

  • COVID-19 Status
  • News
  • Science Highlights
  • Blog
  • Webinars
  • CSP Plans
  • Featured Profiles
  • Careers
  • Contact Us
  • Events
  • User Meeting
  • MGM Workshops
  • Internal
  • Disclaimer
  • Credits
  • Policies
  • Emergency Info
  • Accessibility / Section 508 Statement
  • Flickr
  • LinkedIn
  • RSS
  • Twitter
  • YouTube
Lawrence Berkeley National Lab Biosciences Area
A project of the US Department of Energy, Office of Science

JGI is a DOE Office of Science User Facility managed by Lawrence Berkeley National Laboratory

© 1997-2022 The Regents of the University of California