Skip to content

Commit

Permalink
Misc
Browse files Browse the repository at this point in the history
  • Loading branch information
cmdcolin committed Dec 12, 2024
1 parent dc491e8 commit 5001922
Show file tree
Hide file tree
Showing 16 changed files with 227 additions and 69 deletions.
3 changes: 2 additions & 1 deletion .prettierrc
Original file line number Diff line number Diff line change
Expand Up @@ -2,5 +2,6 @@
"semi": false,
"trailingComma": "all",
"singleQuote": true,
"arrowParens": "avoid"
"arrowParens": "avoid",
"proseWrap": "always"
}
3 changes: 2 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,8 @@

# 0.1.1

- Adds refName to the feature ID to avoid collisions for code that cares about unique ID
- Adds refName to the feature ID to avoid collisions for code that cares about
unique ID

# 0.1.0

Expand Down
75 changes: 51 additions & 24 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,18 +49,26 @@ import NCList from '@gmod/nclist'

### NCListStore

Sequence feature store using nested containment
lists held in JSON files that are lazily read.
Sequence feature store using nested containment lists held in JSON files that
are lazily read.

#### Parameters

- `args` **[object](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Object)** constructor args

- `args.baseUrl` **[string](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/String)** base URL for resolving relative URLs
- `args.urlTemplate` **[string](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/String)** Template string for
the root file of each reference sequence. The reference sequence
name will be interpolated into this string where `{refseq}` appears.
- `args.readFile` **[function](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Statements/function)** function to use for reading remote from URLs.
- `args`
**[object](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Object)**
constructor args

- `args.baseUrl`
**[string](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/String)**
base URL for resolving relative URLs
- `args.urlTemplate`
**[string](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/String)**
Template string for the root file of each reference sequence. The reference
sequence name will be interpolated into this string where `{refseq}`
appears.
- `args.readFile`
**[function](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Statements/function)**
function to use for reading remote from URLs.
- `args.cacheSize` (optional, default `10`)

#### getRegionFeatureDensities
Expand All @@ -69,16 +77,28 @@ fetch binned counts of feature coverage in the given region.

##### Parameters

- `query` **[object](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Object)** 

- `query.refName` **[string](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/String)** reference sequence name
- `query.start` **[number](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Number)** region start
- `query.end` **[number](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Number)** region end
- `query.numBins` **[number](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Number)** number of bins desired in the feature counts
- `query.basesPerBin` **[number](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Number)** number of bp desired in each feature counting bin

Returns **[object](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Object)** as:
`{ bins: hist, stats: statEntry }`
- `query`
**[object](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Object)** 

- `query.refName`
**[string](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/String)**
reference sequence name
- `query.start`
**[number](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Number)**
region start
- `query.end`
**[number](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Number)**
region end
- `query.numBins`
**[number](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Number)**
number of bins desired in the feature counts
- `query.basesPerBin`
**[number](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Number)**
number of bp desired in each feature counting bin

Returns
**[object](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Object)**
as: `{ bins: hist, stats: statEntry }`

#### getFeatures

Expand All @@ -87,8 +107,15 @@ yielding feature objects.

##### Parameters

- `args` **[object](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Object)** 

- `args.refName` **[string](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/String)** reference sequence name
- `args.start` **[number](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Number)** start of region. 0-based half-open.
- `args.end` **[number](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Number)** end of region. 0-based half-open.
- `args`
**[object](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Object)** 

- `args.refName`
**[string](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/String)**
reference sequence name
- `args.start`
**[number](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Number)**
start of region. 0-based half-open.
- `args.end`
**[number](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Number)**
end of region. 0-based half-open.
1 change: 0 additions & 1 deletion index.js

This file was deleted.

5 changes: 3 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -30,15 +30,16 @@
"docs": "documentation readme --shallow src/feature_store.ts --section API",
"clean": "rimraf dist esm",
"prebuild": "yarn docs && yarn clean && yarn lint",
"build:esm": "tsc --target es2018 --outDir esm",
"build:es5": "tsc --target es2015 --module commonjs --outDir dist",
"build:esm": "tsc --outDir esm",
"build:es5": "tsc --module commonjs --outDir dist",
"build": "yarn build:esm && yarn build:es5",
"prepublishOnly": "yarn test --run && yarn build",
"postversion": "git push --follow-tags"
},
"devDependencies": {
"@typescript-eslint/eslint-plugin": "^8.18.0",
"@typescript-eslint/parser": "^8.18.0",
"@vitest/coverage-v8": "^2.1.8",
"cross-fetch": "^3.0.2",
"documentation": "^14.0.1",
"eslint": "^9.5.0",
Expand Down
22 changes: 13 additions & 9 deletions src/feature_store.ts
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,7 @@ export default class NCListStore {

const { histograms } = trackInfo
if (histograms?.meta) {
// eslint-disable-next-line @typescript-eslint/prefer-for-of
for (let i = 0; i < histograms.meta.length; i += 1) {
histograms.meta[i].lazyArray = new LazyArray(
{ ...histograms.meta[i].arrayParams, readFile: this.readFile },
Expand Down Expand Up @@ -159,16 +160,19 @@ export default class NCListStore {
const statEntry = stats.find(entry => entry.basesPerBin >= basesPerBin)

// The histogramMeta array describes multiple levels of histogram detail,
// going from the finest (smallest number of bases per bin) to the
// coarsest (largest number of bases per bin).
// We want to use coarsest histogramMeta that's at least as fine as the
// one we're currently rendering.
// TODO: take into account that the histogramMeta chosen here might not
// fit neatly into the current histogram (e.g., if the current histogram
// is at 50,000 bases/bin, and we have server histograms at 20,000
// and 2,000 bases/bin, then we should choose the 2,000 histogramMeta
// rather than the 20,000)
// going from the finest (smallest number of bases per bin) to the coarsest
// (largest number of bases per bin).
//
// We want to use coarsest histogramMeta that's at least as fine as the one
// we're currently rendering.
//
// TODO: take into account that the histogramMeta chosen here might not fit
// neatly into the current histogram (e.g., if the current histogram is at
// 50,000 bases/bin, and we have server histograms at 20,000 and 2,000
// bases/bin, then we should choose the 2,000 histogramMeta rather than the
// 20,000)
let histogramMeta = data._histograms.meta[0]
// eslint-disable-next-line @typescript-eslint/prefer-for-of
for (let i = 0; i < data._histograms.meta.length; i += 1) {
if (basesPerBin >= data._histograms.meta[i].basesPerBin) {
histogramMeta = data._histograms.meta[i]
Expand Down
4 changes: 2 additions & 2 deletions src/lazy_array.ts
Original file line number Diff line number Diff line change
Expand Up @@ -54,8 +54,8 @@ export default class LazyArray {
for (let chunk = firstChunk; chunk <= lastChunk; chunk += 1) {
chunkreadFiles.push(this.chunkCache.get(chunk, chunk))
}
for (let i = 0; i < chunkreadFiles.length; i += 1) {
const [chunkNumber, chunkData] = await chunkreadFiles[i]
for (const elt of chunkreadFiles) {
const [chunkNumber, chunkData] = await elt
yield* this.filterChunkData(start, end, chunkNumber, chunkData)
}
}
Expand Down
4 changes: 2 additions & 2 deletions src/nclist.ts
Original file line number Diff line number Diff line change
Expand Up @@ -93,8 +93,8 @@ export default class NCList {
}
}

for (let i = 0; i < pendingPromises.length; i += 1) {
const [item, chunkNum] = await pendingPromises[i]
for (const p of pendingPromises) {
const [item, chunkNum] = await p
if (item) {
yield* this.iterateSublist(item, from, to, inc, searchGet, testGet, [
...path,
Expand Down
8 changes: 3 additions & 5 deletions src/util.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,10 @@ import resolve from '@jridgewell/resolve-uri'

export async function readJSON(url, readFile, options = {}) {
const { defaultContent = {} } = options
let str
try {
str = await readFile(url, { encoding: 'utf8' })
return JSON.parse(str)
const str = await readFile(url, { encoding: 'utf8' })
const decoder = new TextDecoder('utf8')
return JSON.parse(decoder.decode(str))
} catch (error) {
if (
error.code === 'ENOENT' ||
Expand All @@ -20,8 +20,6 @@ export async function readJSON(url, readFile, options = {}) {
}
}

export function foo() {}

export function newURL(arg: string, base = '.') {
return resolve(arg, base)
}
8 changes: 4 additions & 4 deletions test/__snapshots__/ensembl_genes.test.ts.snap
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
// Jest Snapshot v1, https://goo.gl/fbAQLP
// Vitest Snapshot v1, https://vitest.dev/guide/snapshot.html

exports[`ensembl genes read with generic-filehandle RemoteFile with http urls small feature queries 1`] = `
exports[`ensembl genes > read with generic-filehandle2 RemoteFile with http urls small feature queries 1`] = `
{
"bins": [
1,
Expand All @@ -22,7 +22,7 @@ exports[`ensembl genes read with generic-filehandle RemoteFile with http urls sm
}
`;

exports[`ensembl genes read with generic-filehandle RemoteFile with http urls small feature queries 2`] = `
exports[`ensembl genes > read with generic-filehandle2 RemoteFile with http urls small feature queries 2`] = `
{
"bins": [
86,
Expand Down Expand Up @@ -1034,7 +1034,7 @@ exports[`ensembl genes read with generic-filehandle RemoteFile with http urls sm
}
`;

exports[`ensembl genes read with generic-filehandle RemoteFile with http urls whole dataset 1`] = `
exports[`ensembl genes > read with generic-filehandle2 RemoteFile with http urls whole dataset 1`] = `
[
[
1,
Expand Down
4 changes: 2 additions & 2 deletions test/__snapshots__/simple.test.ts.snap
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
// Jest Snapshot v1, https://goo.gl/fbAQLP
// Vitest Snapshot v1, https://vitest.dev/guide/snapshot.html

exports[`simple data read with generic-filehandle RemoteFile with http urls 1`] = `
exports[`simple data > read with generic-filehandle2 RemoteFile with http urls 1`] = `
[
[
0,
Expand Down
4 changes: 2 additions & 2 deletions test/ensembl_genes.test.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
//@ts-nocheck
import { RemoteFile } from 'generic-filehandle2'
import { expect, describe, test, afterAll, beforeAll } from 'vitest'
import fetch from 'cross-fetch'
import { RemoteFile } from 'generic-filehandle2'
import NCListStore from '../src/feature_store'
import makeTestServer from './static_server'

Expand Down
2 changes: 1 addition & 1 deletion test/simple.test.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
//@ts-nocheck
import { RemoteFile } from 'generic-filehandle2'
import { expect, describe, test, afterAll, beforeAll } from 'vitest'
import fetch from 'cross-fetch'

import NCListStore from '../src'
Expand Down
6 changes: 3 additions & 3 deletions test/static_server.js → test/static_server.ts
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
const getPort = require('get-port')
import getPort from 'get-port'

const express = require('express')
import express from 'express'

module.exports = async () => {
export default async function staticServer() {
const app = express()
const port = await getPort()
app.use(express.static('test/data'))
Expand Down
5 changes: 2 additions & 3 deletions tsconfig.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,14 @@
"include": ["src"],
"compilerOptions": {
"outDir": "dist",
"types": ["node", "jest"],
"target": "es2020",
"lib": ["dom", "esnext"],
"declaration": true,
"moduleResolution": "node",
"sourceMap": true,
"strict": true,
"noImplicitReturns": true,
"noFallthroughCasesInSwitch": true,
"esModuleInterop": true,
"allowJs": true
"esModuleInterop": true
}
}
Loading

0 comments on commit 5001922

Please sign in to comment.