-
Notifications
You must be signed in to change notification settings - Fork 1
Commit
To get a basic understanding of how Arabic transcription works druid:bc962wz7181 was added to the test suite. The new output directory includes the results `docs/output-2024-07-05`. The two notebooks that analyze word error rates were updated as well.
- Loading branch information
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -15,7 +15,7 @@ The items were exported as BagIt directories from SDR preservation using the [SD | |
So assuming SDR-GET exported the bags to `/path/to/export` and you want rsync just the low service copies to `example.stanford.edu` you can: | ||
|
||
``` | ||
rsync -rvhL --times /path/to/export [email protected]:pilot-data | ||
rsync -rvhL --times --include "*/" --include "*.mp4" --include "*.m4a" --include "*.txt" --exclude "*" /path/to/export [email protected]:pilot-data | ||
``` | ||
|
||
The bags should be made available in a `data` directory that you create in the same directory you've cloned this repository to. Alternatively you can symlink the location to `data` | ||
|
@@ -53,6 +53,14 @@ Install dependencies: | |
$ pip install -r requirements.txt | ||
``` | ||
|
||
To run the AWS and Google tests you'll need to: | ||
|
||
``` | ||
$ cp env-example .env | ||
``` | ||
|
||
And then edit it to add the relevant keys and other platform specific configuration. | ||
|
||
## Run | ||
|
||
Then you can run the report: | ||
|
@@ -75,14 +83,6 @@ To run the unit tests you should: | |
$ pytest | ||
``` | ||
|
||
If you want to run the AWS and Google tests you'll need to: | ||
|
||
``` | ||
$ cp env-example .env | ||
``` | ||
|
||
And then edit it to add the relevant keys and other platform specific configuration. | ||
|
||
## Analysis | ||
|
||
There are some Jupyter notebooks in the `notebooks` directory which you can view here on Github. | ||
|
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
{"results": [{"alternatives": [{"transcript": "okay good thank you so much Mark my name is Amanda Whitmire I am from the Miller Marine Biology library or Stanford libraries and I'm going to spend the next ten minutes talking about student work and FERPA and I nixed copyright cuz I only have 10 minutes and there's a copyright person coming up in a bit later so we won't worry about that for now so for but is US law that I'm discussing it through the lens of a librarian at a marine research station with the intent of a really just putting it on our Collective radar as we move forward talking about archives and access over the next couple days and just for context my library is actually located at the Hopkins Marine station which is about 90 miles south of here perched on the southern tip of Monterey Bay", "confidence": 0.97445846}], "resultEndTime": "69.700s", "languageCode": "en-us"}, {"alternatives": [{"transcript": " lucky me", "confidence": 0.8633474}], "resultEndTime": "71.020s", "languageCode": "en-us"}, {"alternatives": [{"transcript": " so let's just get right into it what is FERPA so Rupa is federal law that came in response really to a history of inconsistent institutional policies and improper disclosure of student information so the intent of her power really is to protect privacy of both parents and student information", "confidence": 0.9288033}], "resultEndTime": "91.890s", "languageCode": "en-us"}, {"alternatives": [{"transcript": " and what purpose says is that institutions that receive federal funds have to provide parents with access to educational records of their children although this right transfers to the student once they turn 18", "confidence": 0.9748582}], "resultEndTime": "104.990s", "languageCode": "en-us"}, {"alternatives": [{"transcript": " it prohibits releasing the educational records of students other than directory information without written consent", "confidence": 0.9647188}], "resultEndTime": "113.700s", "languageCode": "en-us"}, {"alternatives": [{"transcript": " although consent is not required for release of Education records to certain institutions and organizations", "confidence": 0.9209177}], "resultEndTime": "121.650s", "languageCode": "en-us"}, {"alternatives": [{"transcript": " which is slightly confusing right so recall that I mentioned that directory information can be released without consent so what is directory information it's things that wouldn't be considered a violation of the student student privacy which surprisingly includes things like place and date of birth or photograph if they're on an athletic team you can release their height and weight so leave it to you to decide how invasive you think these this non private information really is", "confidence": 0.95242727}], "resultEndTime": "150.420s", "languageCode": "en-us"}, {"alternatives": [{"transcript": " and then recall also that consent wasn't required to release certain education records of students what are those four but defines those as being student works that are directly related to the students process through school and they're maintained by the educational institution and assignments digital media so already this is pretty confusing and as Librarians and archivists you're probably wondering what what is the overlap between student work you might have in your collection and FERPA what is considered an education record and might have to be collected and what is something that we can make available and so a l a i actually asked the US Department of Education this question in 1993 by saying something like a thesis which is student gives to the library generally for the known purpose of being made available as a research material the process of the student giving", "confidence": 0.971825}], "resultEndTime": "210.830s", "languageCode": "en-us"}, {"alternatives": [{"transcript": " their material to the library for fire to form of tacit permission to make those materials available as research objects", "confidence": 0.852795}], "resultEndTime": "217.140s", "languageCode": "en-us"}, {"alternatives": [{"transcript": " so as we've seen the regulations concerning student work or educational records can be confusing but students is important in our understanding of her present because I think students produce really important research that's critical to our understanding of the world and to the extent possible I want to be able to make those research materials available and so for the next few minutes I'm going to give you an example of something that I've been navigating through my work recently to give you some context on on my thoughts on this issue and it's it's going to seem like a little bit of a tangent at first but stick with me it's all going to it's all going to come back together", "confidence": 0.9705823}], "resultEndTime": "256s", "languageCode": "en-us"}, {"alternatives": [{"transcript": " so has anyone in this room ever used the Merlin Bird ID app or inaturalist to identify something yes my people my people perfect so when you take a picture of something and you uploaded into Merlin or two I naturalist what you're doing is you're collecting three pieces of information I saw this thing in this place at this time and those three pieces of information together are called a species occur in", "confidence": 0.9517811}], "resultEndTime": "281.120s", "languageCode": "en-us"}, {"alternatives": [{"transcript": " and there's a place called jebus a global biodiversity information facility that gathers species occurrence records from inaturalist from eBird from academic research so it's happening all over the world and as it stands they have almost 1.4 billion records of I saw this thing in this place at this time and why is that important well if you're interested in understanding things like how climate change might be Shifting the distribution of animals and plants on the planet you have to understand not only where they are now but where they were before and that is the power of 1.4 billion observations in Jeep", "confidence": 0.9685778}], "resultEndTime": "319.220s", "languageCode": "en-us"}, {"alternatives": [{"transcript": " so as one example so you're interested in knowing where this very common read an enemy can be found in the world and feel Clara and you go to jebus and you take a look and see what they have you'll see that there's almost 2,500 BCE the current records which sounds like a lot of data", "confidence": 0.9395002}], "resultEndTime": "335.810s", "languageCode": "en-us"}, {"alternatives": [{"transcript": " but if you take one step deeper into the data what you find is that the large majority of our records of the organism and in fact the majority of all of the records in Jesus are very recent and if you're interested in looking at things on the scale climate change you need to be able to look much further back in time so how do we feel this observational Gap and that's where I think long-standing educational institutions like Stanford and others really are poised to make a significant contribution and one area where I think we can make a contribution locally is through the undergraduate work of our students do in particular I just want to give this one example of two students Sarah Gilman and Instagram that came to Hawkins in the summer of 1993", "confidence": 0.9625333}], "resultEndTime": "376.840s", "languageCode": "en-us"}, {"alternatives": [{"transcript": " and what they chose to do further research project was to resample a transect that was first sampled in 1931 and 1933 so there's an intertidal line that goes from the shore out 105 yards they finally found the line after 3 days started by that Brass Bull bear looked at 19 square yards along this line they counted up all the species they compared with a saw to what another researcher had seen back in the 1930s and what they discovered was that there was a shift toward warmer water species and along with the temperature time series that we also maintained at Hopkins they hypothesized that this might be due to shifting ocean temperatures", "confidence": 0.9550408}], "resultEndTime": "412.250s", "languageCode": "en-us"}, {"alternatives": [{"transcript": " interesting paper is in my library", "confidence": 0.84562236}], "resultEndTime": "415.670s", "languageCode": "en-us"}, {"alternatives": [{"transcript": " So based on that work they came back the following summer they did more work they identified over 58,000 individuals from 105 different invertebrate taxa and again they were able to with this greater body of data actually show a strong correlation between species shifts and changing ocean temperatures and their work as undergraduates was published in science not bad", "confidence": 0.9659916}], "resultEndTime": "440.750s", "languageCode": "en-us"}, {"alternatives": [{"transcript": " but their work is just one paper and a much larger collection that we have in our library that expands from every year of class was the same class was taught from 1963 to 2011 and these are all of their Unbound papers which are now actually here on campus being digitized", "confidence": 0.9428387}], "resultEndTime": "457.440s", "languageCode": "en-us"}, {"alternatives": [{"transcript": " but what you also find out is that my collection student papers is not unique so these are similar collections in Washington of it Friday Harbor auction starts in the 40s up at Bodega Marine Labs just north of us here the collection starts in the 1920s and is on both of these are ongoing there's a collection from Banfield Marine Science Center up in Canada versus Colombia", "confidence": 0.94704485}], "resultEndTime": "477.910s", "languageCode": "en-us"}, {"alternatives": [{"transcript": " and when I reached out to my colleagues and other marine stations what I found is there are literally thousands of undergraduate student research papers available for local use in our libraries so if you're interested in exploring something like climate change for an intertidal species this body of work is unprecedented and its coverage in space and time", "confidence": 0.95121855}], "resultEndTime": "499.060s", "languageCode": "en-us"}, {"alternatives": [{"transcript": " but what does this have to do with four but you're all wondering Amanda and so what I discovered when I was working with my colleagues up and down the coast to assess the potential for creating just a Federated catalog let's get all the bibliography of all the students papers together put them in one place with a researcher wanted to know just what's out there in terms of student research can I find it what I discovered is that in at least two cases of those libraries on the last slide", "confidence": 0.9659059}], "resultEndTime": "525.070s", "languageCode": "en-us"}, {"alternatives": [{"transcript": " the Librarians were limited in their ability to share bibliographies because of the institutions understanding a FERPA so in the case in front of you hear the librarian was told she had to remove all of the student names the student author names from the papers from the movie graffiti this is our oldest collection it goes back to 1928 so she had to remove all those names by hand he was also told she couldn't put these in the library catalog so what she did is she she created a PDF and she shares this on the library web page and if you'd like to search it you can use the control left to search her catalog of student papers and another case the librarian was told point-blank if you can't put this bibliography online. It's an invasion of students privacy", "confidence": 0.96629226}], "resultEndTime": "565.500s", "languageCode": "en-us"}, {"alternatives": [{"transcript": " do you remember what the US Department of Education said when a student gives the work to the library it's tacit permission to share the work I'm not talking about copyright that's a totally separate issue but it's perfectly acceptable to make the work available through the library and their name is directory information that's not protected there seems to be a lot of confusion about 4 but that's really limiting our ability is Librarians to distribute, and make it discoverable and ways that we think are an acceptable", "confidence": 0.95877695}], "resultEndTime": "594.030s", "languageCode": "en-us"}, {"alternatives": [{"transcript": " so I hope I hope you've been able to show you that there's potential that lies within student work and the impact that you can have that you might understand my frustration and my interest in for both as a librarian I feel a moral obligation to make critical observational research data and information available to research the researchers to the widest extent possible under the growing threat of climate change the dogs or worse yet catalogs that you can't even find are completely unacceptable as a status quo and I know I don't need to tell you guys that I'm so we need new ways to discover and access the research hiding in our Collections and in the case of student works this means that we have to understand a limit and the latitudes associated with FERPA and that's why I wanted to share it with you today thank you", "confidence": 0.9727089}], "resultEndTime": "643.650s", "languageCode": "en-us"}], "totalBilledTime": "644s", "requestId": "243343919523683943"} |