nulib / avalon Goto Github PK
View Code? Open in Web Editor NEWThis project forked from avalonmediasystem/avalon
Variations-on-Video Hydra app
License: Apache License 2.0
This project forked from avalonmediasystem/avalon
Variations-on-Video Hydra app
License: Apache License 2.0
Based on data from the pilot, come up with a cost breakdown. In addition, we should verify that there are no more cost-saving measures we should be taking (reserved instances/ glacier or reduced redundancy storage for certain files / smaller instances based on load_
Provisioning the Avalon in AWS environment is handled via cloud formation scripts. It is critical the Devops team have time to understand how the scripts work and what they do. MBK will give an overview, and point to any training.
Customize application header and footer to use NU's branding header and footer templates.
@mbklein sets up a meeting with RDC/Devops to go over infrastructure, published cloud formation scripts, and next steps needed (represented in other tickets)
The goal of the meeting is for RDC/Devops to finalize the plan for the pilot and create any new issues/ carve out time for training if necessary.
Topics to be covered:
So my thoughts are we need three main components: Player, ActiveEncode (for derivatives), and ActiveStreaming. We'd also have enriched metadata as a need, but it's not minimum out the door. We can add that in as we go given our current Avalon instance doesn't have particularly rich metadata.
This would look something like:
ActiveEncode
Player
ActiveStreaming
Anything else NU needs? We can probably then evolve these into larger tickets on the main Avalon board. On our end I'm seeing a workflow of:
Use NU template page layout types to display content.
2.4 tb is currently being eaten up by matterhorn files which can be cleaned up per conversation in #4 .
Please run investigate and run
https://wiki.dlib.indiana.edu/display/VarVideo/Maintaining+Matterhorn+Disk+Space
per MBK.
Create a Northwestern look & feel to NU's Avalon application by using NU template Sass files.
Update Avalon's default search styling to NU branding. Verify default search functionality works.
@mbklein @adamjarling has started to work on customizing Avalon 6.x for NU. We need a repo for him to push to (and you to pull from ) for http://avalon.repo.rdc.library.northwestern.edu.
Can you take this and run with it?
Hi,
Can I have access to libwowza1p.library.northwestern.edu
? I'd also like to know what you actually it when you SSH into it as I can't even get a response from it when on the SSL VPN. I don't need any special power perks but I do need to be able to cat logs, etc.
Thanks!
Hi Nicole,
I need to add Sharaku https://avalonweb2p.library.northwestern.edu/media_objects/numedia:47291 to Avalon for a class, and I’m getting the below errors when I Import the metadata (ALMA # 21678441120002441). Could this be fixed as soon as possible? I can’t seem to add Access Control for the class until these Resourse Description errors disappear. Please let me know.
Thanks!
Molly
Update NU Avalon homepage to reflect NU branding, and to incorporate 'Browse' functionality.
I’m loading the episode 2 file from my local drive to that Alps dropbox now – it should take about 45 minutes to finish loading, then I’ll try uploading to our Avalon item again. It does now appear when I Open Dropbox in the Avalon item, so looks like this is going to work – yay! Thank you, Michael!
Molly
From: Michael B. Klein
Sent: Friday, January 06, 2017 1:10 PM
To: Main Library Reserves [email protected]; Debs Cane [email protected]; Nicole Gabrielle Finzer [email protected]
Cc: Alex Barontini [email protected]; Kurt I Munson [email protected]
Subject: Re: adding 2 files as sections to an existing “Course Media” item in Avalon – for a Monday 1/9/17 Course Reserves request
Try the Alps dropbox directory called “Course_Media,_Northwestern_University_Libraries”
mbk
From: Main Library Reserves [email protected]
Date: Friday, January 6, 2017 at 11:48 AM
To: Debs Cane [email protected], Nicole Gabrielle Finzer [email protected], "Michael B. Klein" [email protected]
Cc: Alex Barontini [email protected], Kurt I Munson [email protected]
Subject: adding 2 files as sections to an existing “Course Media” item in Avalon – for a Monday 1/9/17 Course Reserves request
Hi, Debs, Nicole, and Michael,
Prof. Michael Kramer has asked to have episodes 2 & 3 of Jazz added to one of his Winter Quarter Canvas sites by this Monday, the 9th. I’ve captured both and have placed the files in Alps, Course Video. They are:
21448275530002441_jazz_e2_gift.mov
21448275530002441_jazz_e3_ourlang.mov
Ideally, we’d add these as sections to our already existing Avalon item for the Jazz series – see https://avalonweb2p.library.northwestern.edu/media_objects/numedia:17994. However, because episodes 1 and 4 through 9 were added to Avalon as part of the Course Media collection, I’m unable to add episodes 2 and 3 to the same item in Avalon. I tried adding the episode 2 file to Course Media in Alps to see if that might work. I was able to place it there, but then it doesn’t appear when I Open Dropbox in our Avalon item. Can this be fixed so that these 2 episodes can be in the same Avalon item as the other episodes? If so, could it be done sometime today to meet our Monday deadline? Please let us know as soon as possible.
Thanks,
Molly
Molly Zolnay
Library Assistant
Northwestern University Libraries
Northwestern University
www.library.northwestern.edu
(847)467-1985
Update Avalon sidebar to reflect NU sidebar branding.
Move production Avalon Derivates to AWS bucket in preparation for cut over. We will need to mark time that this occurs as well, so we can grab any diffs production or course reserves works on in the meantime.
As we move forward with the Avalon on AWS pilot, we need a plan for monitoring. If systems fail, people will need to be notified and a call tree will need to be set up to bring the system back up. OPSgenie should get notifications.
Opsgenie integration with AWS Cloudwatch.
https://www.opsgenie.com/docs/integrations/aws-cloudwatch-integration
When things fail we need to know who to call when. This is somewhat related to #21.
Train Cecile and Molly on features of the new system. Of note, playlists and markers, fast uploads and batch ingestion.
Currently we're unclear on how security patches will work in an AWS cloud environment (burn instance and bring it back up, patch hot, etc). In particular do we have a plan for SOLR ( @mbklein ), etc. If we can burn things down and bring it back up, it should be OK.
Make the NU Avalon responsive initially without major structural updates to the Avalon templating.
Note: There is only so much which is possible give Avalon's default markup. The scope of this ticket is to fix/update any UI issues which are major after applying NU brand assets.
Use Northwestern favicon instead of Avalon favicon
We need a new testing script for Avalon 6.0 - below is the one used for the Avalon project. We need to alter for local testing.
Using the spreadsheet below, please complete assigned sections and kick off bugs.
And save it to Avalon Test Scripts/completed_tests using YYYYMMMDD_lastname as a filename.
For each bug found, add a new github issue to avalon and label it "bug".
Bugs will be triaged and dealt with as necessary.
Please put completed test script in completed tests
FILL IN SERVER NAME
Hi all, I section 1 script and notes in: https://docs.google.com/spreadsheets/d/1v-K4NWEPyERabHHCMPADI79P-d1FAu2_fqy_dgfngdI/edit#gid=1730858809
delete as necessary if not doing full testing. .
Michael found 1800+ audio files that did not get moved to farchive during his audit. He will generate a report with file name and collection for DCM team to determine if there are records in AVR (spot checking makes sense) or if they were part of a failed batch and re-ingested, or if they were test/old files and not supposed to be moved. related to issue #12 now closed
Set up a local instance for development.
Is the production site running?
Are staff blocked from performing their work?
Erin cannot access alps.library.northwestern.edu/dropbox
Publish out the network diagram for Avalon in AWS. Just attach to this issue. I'll put it in the wiki.
My understanding (correct me if I'm wrong @mbklein) is that right now you need a VPN to connect into AWS machines. To load the data into the Fedora up on AWS, I assume we don't want to install a VPN on avalon-prod, rather we want to either set up some kind of keypair or else temporarily expose our Amazon Fedora to avalon-prod while we push data.
We need ssh (and root) access to avalonweb2p, thanks!
In order to hold down costs, we should set the derivatives storage to reduced redundancy. These are easily regrettable files, and even at this level are fairly durable.
This is our first crack at this. We will have lots of room for improvement. The goal is to develop a list of action items to take after/ during the pilot to:
When a user goes to log in, the user is returned to the home screen not logged in, and has to log in once more in order to be logged in.
New collection added to the HP of media.library.northwestern.edu
As I mentioned yesterday to Michael and Chris, I have been working on an audio collection of interviews with African authors from the Herskovitz Library. It’s nearly ready to be imported to AVR, and I was wondering if it would be possible to have a link to it on the homepage like the Marcellus and Football collections. This collection is significant, and I think it would score a lot of points with Esmeralda to have be quite visible. Michael and Chris told me that it’s possible to adjust that page however we want – I’m hoping that it’s an easy task and that there’s time for one of the developers to do it.
Let me know if it’s feasible, and what the time frame would be. I am just waiting for Jen or Nicole to sign off on the metadata to import the collection.
Thanks!
Laura
We need to either create a new GH repo or update the current 4.x gh repo/fork (this one here!). This should be done ASAP so Adam can start to work on customizing avalon.
Is the production site running?
Are staff blocked from performing their work?
After a successful batch ingest, media files are not being automatically removed from the folder on alps.
Media files removed from folder upon successful batch ingest.
Media files remain in alps folder after successful batch ingest
Folders with this issue:
@carrickr can you add a link or drop in the thoughts you wrote up directly here?
Before go-live we should load-test avalon on AWS.
We may switch from LDAP to NUIT's OATH for our sign in on Avalon. Per @mbklein "I'd rather be dealing with tokens than firewalls"... The end goal is that authentication into Avalon works as expected for an end users : I can log in to Avalon using my netID and pass.
It seems like we have several very large derivatives (derivatives that are larger than masters). We should evaluate and retranscode as necessary. This ideally should take place before cutover.
Nicole and David out June 5-9. Nicole out June 12 week as well.
INTERCESSION June 12-16
Classes Start June 19
Give a report to CC about files moved, data in .
It seems for the last quarter we only increased our storage by ~2tb. Which seems considerably lower than it should be. @mbklein to investigate.
@d-venckus can you do a report on our isilon storage and attach to this ticket?
During this cycle, the rdc team was largely heads down investing, drafting the AWS pilot proposal (https://github.com/nulib/repodev_planning_and_docs/wiki/Avalon-AWS-Pilot-%28Summer-2017%29), and ensuring we could pull this off.
Discussion with outside folks after a call was made that we could hit our summer session deadline.
During this period we will be moving data and swill start testing/ train users.
Testing and final migration (if necessary)
NOTE: DS will be in Italy, Nicole is at a conference. @jenyoung is in charge.
Intercession. During this week we will do final production testing, ensuring that canvas is working, etc. In addition, we will ensure that all of our users that need access have it.
Master files in Avalon should sync to local storage during the pilot phase. We have not made a decision about long term storage of files, and it is out of scope for this project. For now we want to have a copy of renamed master files (not derivatives) synced to local storage.
I went thru my initial assessment again comparing what had been removed after the script was made to work again on Alps. There are still files that should be removed on Alps. I do not know if they are in the farchive.
Prof. Kaveh Askari is currently teaching 2 classes in Qatar (2017SP_MIT_351-0_SEC70 and 2017SP_MIT_398-0_SEC71) and has encountered the following issue with the playback of Avalon video titles for his classes…
If a video has been paused for about 10 to 15 minutes, he is unable to resume playback and the browser becomes unresponsive. This happens with both Chrome and Safari.
Kaveh reports that this has been the case with all his Avalon video titles so far. They are…
For 2017SP_MIT_351-0_SEC70:
· Alexandria Again and Forever
· Cairo Station
· Dry Summer
· Egyptian Story
· Rana’s Wedding
· Wadjda
· Waltz with Bashir
For 2017SP_MIT_398-0_SEC71:
· Battle of Algiers
· Black Girl
· District 9
· The Good, the Bad, the Weird
· Hour of the Furnaces
· The Music Room
· Night Cries
· Welcome to Nollywood
· The World
I’ve asked him to give Firefox a try, and suggested refreshing the page, etc. (See below correspondence for more details).
Might there be some way of preventing this playback issue from occurring? Would you please investigate and get back to Kaveh?
Many thanks, and all best,
Molly
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.