Archive Update

  • AO3 site slowness & known issues

    By Claudia Rebaza on Thursday, 21 November 2013 - 12:47am
    Message type:
    Список тегов:

    Banner by Diane with the outlines of a man and woman speaking with word bubbles, one of which has the OTW logo and the other which says 'OTW Announcement'

    We recently deployed new code, which fixed a couple of very old bugs and introduced improvements to the kudos feature. Behind the scenes, we've been working on setting up new servers and tweaking server settings to make everything run a little more smoothly during peak times. The end of the year (holiday season in many parts of the world) usually means more people with more free time to participate in more challenges, read more fic, or post more fanart, resulting in more site usage. Continue reading on AO3 News

  • Archive of Our Own Newsletter - March/April 2013

    By Camden on Tuesday, 30 April 2013 - 1:34am
    Message type:

    March and April were busy busy busy months for the Archive. Our output included, among other things, a new header, an updated roadmap, and our annual April Showers promotion, a joint effort between Fanlore and the AO3 to highlight the fandoms you love!

    All the Archive news that's fit to print!

    As part of the continuing discussion about the OTW's decision to include meta, or fandom nonfiction, on the Archive, the OTW's Board released a statement. This statement outlined the history of the decision as well as the Board's reasons for including fandom nonfiction on the Archive. Another post, which was the culmination of a discussion between several committees, further clarified the position of the Archive's thoughts on fandom nonfiction and tentatively outlined the steps that would be necessary in introducing fandom nonfiction into the Archive.

    The OTW Board released an updated Archive Roadmap. The Roadmap outlines future improvements and changes of the Archive. It is maintained by the Accessibility, Design & Technology committee, though it was created with the input of several other committees including Abuse, Internationalization & Outreach, Open Doors, Support, Systems, Tag Wranglers, and Translation. If you're interested in the future of the Archive, go check it out!

    Although the event was not run by Archive committees (big props to the Development & Membership committee for all their hard work!), AO3 was happy to host a banner for the OTW's annual April Membership Drive as well as contributing a post sharing how much it costs to run the Archive. This year's drive was the most successful drive in OTW history. Thank you to everyone who donated!

    Release 0.9.6 went live and what a release it was! It presented us with, most notably, a new header and new notification emails. Other new features included the ability to disable guest comments, and a new and improved front page. This release fixed several bugs, updated Tag Wrangling and Admin features, and gave us several other goodies. Unfortunately, this release also presented some unanticipated problems for several users. Our known issues posts describes these issues and we encourage any users who are encountering problems with Release 0.9.6 to look at the above post or contact Support.

    Fanlore and AO3 ran their annual April Showers promotion, showcasing fandoms on AO3 and on Fanlore. The Archive's tumblr ao3org with the Fanlore twitter highlighted a fandom a day for the month of April.

    Adventures with Support

    In the months of March and April so far, we've managed 798 unique tickets so far - and we still have a couple days to go!

    AD&T Committee business of note

    In addition to releases, we've been working on documentation to start recruitment for selected positions soon.

    Support Committee business of note

    We will be holding another Open Chat session with the Support Staff in our public chat room (the link will be made available on the day of the chat) this coming Saturday, May 04, 2013 at 16:00:00 UTC lasting through this Sunday, May 05, 2013 at 04:00:00 UTC. Members of Support will be available to interact with users one-on-one in live chat. See what time that is where you live. (Real Life Monsters ate the April chat.)

    Tag Wrangling Committee business of note

    Tag Wrangling opened for volunteers and had such an overwhelming response that we had to hit the pause button long enough to catch up! We've now inducted the over 50 (\o/) new volunteers, and should be re-opening recruitment soon. In concert with that, staff have been hard at work putting together a comprehensive training plan; our basic tutorials are already completed and in use, and more advanced instruction is on the way. Meanwhile our new wranglers have enthusiastically been helping all our experienced hands in cleaning up tags in fandoms across the Archive; we're looking in better shape than ever!

    And finally...

    We are floored by the generosity of the participants in AO3 Auction. From the bottom of our squeeing hearts: THANK YOU!

    Questions? Comments?

    We welcome feedback from users! If you have questions or comments, feel free to leave them in the comments of the latest news post, or send in a Support request (if you're reporting a bug, please send that to Support, as they're super efficient - comments on our news posts sometimes get overlooked).

    Mirrored from an original post on AO3 News.

  • Archive Of Our Own Newsletter - January/February 2013

    By Camden on Sunday, 17 February 2013 - 7:40pm
    Message type:

    Welcome to the January-February Newsletter! We hope you had a good holiday season and are having a happy new year! Despite the term break in December, January was busy busy busy with upgrades and releases for all. February is also turning out to be a big month with releases, spotlights, tag wrangling fixes, and header posts galore! Here's what we've been up to:

    All the Archive news that's fit to print!

    2012 was full of Archive milestones. In November we passed the 500,000 work mark in 10,000 fandoms. In December, the Archive passed 100,000 users. Check out this post for further milestones that we passed in 2012.

    Tag Wrangling shared their process for wrangling Additional Tags and why Additional Tags are not as un-wranglable as one might think.

    Mini-Release 0.9.4 went live with a small number of bug fixes. We were also very happy to bring back invitation requests in December. Release 0.9.5 and its redux went off without a hitch.

    Fandom Tags are now alphabetized regardless of articles. Wranglers now have the ability to assign a sort name different from a display name, making it easier for us to wrangle and browse fandoms!

    We recently posted a Spotlight on Systems Committee. If you've ever wondered what goes on behind the scenes to support the technical systems of the Archive, read all about it here.

    AD&T are hard at work on a redesign of the site header. Check out our little preview and tell us what you think.

    And finally, the Board approved meta hosting on the Archive. Please see this post for details on how this decision affects you, the Archive, and the Archive staffers.

    Adventures with Support

    Things are keeping busy in the Support world. We've got a new co-chair, and the training is letting us solidify all of our training. We're looking forward to a solid year with proactive communication with both our fellow committees and our users!

    Open Doors Update

    Open Doors is still working toward an automated import for the 852 Prospect Archive and recently held two open house chats (read more here). In the meantime, we've opened up manual importing by inviting all 852 Prospect authors to the Archive. Check out Open Doors' post for further instructions on manually importing works from that particular Archive.

    AD&T Committee business of note

    We're excited about the upcoming year and are looking forward to everything we'll be sharing with you. On a more serious note, we recently reviewed our emergency plan in the event that our site is compromised and requires an emergency shutdown.

    Support Committee business of note

    Support will be hosting a Live Chat February 23rd-24th, from 4pm to 4am UTC (What time is that for you?) As Support, our function is to help users with bugs and issues, and pass reports on to our Coders and Systems team, who actually keep the place running. So, if you have questions or comments about AO3 or OTW policies, good or bad, Support Chat isn't the right place for them. If you do want to talk to someone about policy issues (meta on the Archive, philosophical issues with the tagging system, category change, etc.) we can direct you to the appropriate admin post or contact address so you can leave feedback directly for the people dealing with the area of your concern.

    Tag Wrangling Committee business of note

    We've clarified some major weirdness in our guidelines regarding AU tags and inconsistencies regarding the canonicals for Original Characters in Relationship tags.


    We've had some scheduled downtime during the past couple of weeks. Each time was for two upgrades and some site maintenance to build a better Archive. Apologies to anyone who was inconvenienced by this! We also received reports from some users that Avast was sending out malware warnings when users tried to access the Archive. The cause of this malware warning was external and no cause for alarm.

    Questions? Comments?

    We welcome feedback from users! If you have questions or comments, feel free to leave them in the comments of the latest news post, or send in a Support request (if you're reporting a bug, please send that to Support, as they're super efficient - comments on our news posts sometimes get overlooked).

    Mirrored from an original post on AO3 News.

  • AO3 site security (constant vigilance!)

    By .Lucy Pearson on Thursday, 7 February 2013 - 10:30pm
    Message type:
    Список тегов:

    While developing the Archive of Our Own, site security is one of our top priorities. In the last couple of weeks, we've been reviewing our 'emergency plan', and wanted to give users a bit more information about how we work to protect the site. In particular, we wanted to make users aware that in the event of a security concern, we may opt to shut the site down in order to protect user data.


    Last week we were alerted to a critical security issue in Ruby on Rails, the framework the Archive is built on. We (and the rest of the Rails community) had to work quickly to patch this hole: we did an emergency deploy to upgrade Rails and fix the issue.

    As the recent security breach at Twitter demonstrated, all web frameworks are vulnerable to security breaches. As technology develops, new security weaknesses are discovered and exploited. This was a major factor in the Rails security issue we just patched, and it means that once a problem is identified, it's important to act fast.

    Our security plans

    If the potential for a security breach is identified on the site, and we cannot fix it immediately we will perform an emergency shutdown until we are able to address the problem. In some cases, completely shutting down the site is the only way to guarantee that site security can be maintained and user data is protected.

    We have also taken steps for 'damage limitation' in the event that the site is compromised. We perform regular offsite backups of site data. These are kept isolated from the main servers and application (where any security breach could take place).

    In order to ensure the site remains as secure as possible, we also adhere to the following:

    • Developers are subscribed to the Rails mailing list and stay abreast of security announcements
    • We regularly update Rails and the software we use on our servers, so that we don't fall behind the main development cycle and potentially fall afoul of old security problems
    • All new code is reviewed before being merged into our codebase, to help prevent us introducing security holes ourselves
    • All our servers are behind firewalls
    • All password data is encrypted

    What you can do

    The main purpose of this post is to let you know that security is a priority, and to give you a heads up that we may take the site down in an emergency situation. Because security problems tend to be discovered in batches, we anticipate that there is an increased risk of us needing to do this over the next month. In this case, we'll keep users informed on our AO3_Status Twitter, the OTW website and our other news outlets.

    Overall site security is our responsibility and there is no immediate cause for concern. However, we recommend that you always use a unique username / password combination on each site you use. Using the same login details across many sites increases the chance that a security breach in one will give hackers access to your details on other sites (which may have more sensitive data).

    We'd like to thank all the users who contacted us about the latest Rails issue. If you ever have questions or concerns, do contact Support.

    Mirrored from an original post on the Archive of Our Own.

  • Archive of Our Own Scheduled Downtime: Firewall upgrade

    By .Lucy Pearson on Wednesday, 16 January 2013 - 11:29am
    Message type:
    Список тегов:

    The Archive of Our Own will have some scheduled downtime on Thursday January 17 at 18.30 UTC (see what time this in in your timezone). We expect the downtime to last about 15 minutes.

    This downtime is to allow us to make some changes to our firewall which will make it better able to cope under heavy loads. This will help with the kinds of connection issues we experienced last week: our colocation host has generously offered to help us out with this (thanks, Randy!).

    As usual, we'll tweet from AO3_Status before we start and when we go back up, and we'll update there if anything unexpected happens.

  • Planned Archive downtime & maintenance: new servers are coming!

    By .Lucy Pearson on Monday, 7 January 2013 - 5:34pm
    Message type:
    Список тегов:

    The Archive will be down for maintenance for short periods on 8, 10 and 11 January. The maintenance is scheduled to start at approximately 05.15 UTC on each day (see what time that is in your timezone), and will last less than an hour each time. We'll put out a notice on our Twitter AO3_Status when we're about to start.

    Downtime details

    8 January 05.15 UTC: c. 15 minutes downtime.

    10 January 05.15 UTC: c. 25 minutes downtime.

    11 January 05.15 UTC: c. 50 minutes downtime

    What we're up to

    The Archive has grown massively over the past year - during the first week of 2013 we had over 27.6 million pageviews! To cope with the continuing growth of the site, we're adding three more servers. We're also reorganising the way our servers are set up to ensure that they're working as efficiently as possible, and to make it easy for us to add more machines in future.

    Our colocation host installed the new machines in late December. We're now moving over to using them, and reorganising our setup. We're doing the work of moving over to our new database server in several small chunks, which will keep downtimes short and make it easier for us to identify the source of any problems which may arise.

    What's next?

    Once this has been done we'll deploy the Archive code on the new servers and test it out. We'll be looking for some help with this - stay tuned for another post.

    When we're happy that everything is working right, we'll make the switch to using the new servers. No downtime expected at present, but we'll keep you posted if that changes.


    Thanks for your patience while we work.

    We're able to continue expanding the Archive and buying new hardware thanks to the generosity of our volunteers, who give a great deal of time to coding and systems administration, and of OTW members, whose donations pay for the Archive's running costs. If you enjoy using the Archive, please consider making a donation to the OTW. We also very much welcome volunteers, but are currently holding off on recruiting new volunteers while our lovely Volunteers Committee improve our support for new volunteers (we'll let you know when we reopen). Thank you to everyone who supports us!

    Mirrored from an original post on the Archive of Our Own.

  • No cause for concern: malware warnings on the AO3

    By .Lucy Pearson on Friday, 4 January 2013 - 4:28pm
    Message type:
    Список тегов:

    A number of users have reported receiving malware warnings from Avast when accessing the Archive of Our Own. We haven't been hacked, and there is no cause for concern - the warning was a false positive.

    Avast is erroneously flagging a file used by New Relic, which we use to monitor our servers (you can see more details in (this thread). New Relic are working with Avast to resolve the issue, and we expect things to be back to normal very shortly (we have had only a small number of reports today).

    Thank you to everyone who alerted us to this! If you see something unexpected on the site, we always appreciate hearing about it right away. You can keep track of the latest site status via our Twitter AO3_Status, and contact our Support team via the Support form.

  • AO3 invitation requests are back (just in time for holiday gifting)!

    By .Lucy Pearson on Tuesday, 18 December 2012 - 11:53am
    Message type:
    Список тегов:

    It's the season of giving! So, we're pleased to announce that invitation requests are back on the AO3!

    Once upon a time (i.e. six months ago), users with Archive accounts could request a few invitations to give out, allowing them to share the Archive with friends and help form communities of like-minded fans.

    Unfortunately, earlier this year, as many of you may remember, the Archive was having serious performance issues (we saw the sad 502 page far too often). While our coders and systems team hurried to implement emergency fixes, it was decided that we needed stricter control of the number of accounts being created to reduce the likelihood of unexpected overload. (Generally, people browsing the site without being logged in put a certain amount of stress on the servers, but it's the account perks like bookmarking, subscribing, and accessing a full reading history that contribute to server load to a larger degree.) Back then only 100 invitations were issued to people in the queue each day, so additional user requests could have a serious impact! So, in June, the difficult decision was made to stop giving invitations to existing users. You can read more about what was going on then in our post, Update on AO3 performance issues.

    Over the next five months our software upgrades and code improvements caught up with the demand. The queue rate was increased several times, most recently to 750 invitations per day. Given that, we've wanted to go back to giving out invitations to existing users, but there were a few issues to be resolved before we could start.

    First, the request form had to be altered to set a maximum number of invitations that a user can request at once. Second, the 1,200 user requests that were in the list when it was shut down had to be addressed. Since we had no limits on how many invitations could be requested back then, we had quite a few requests for very large numbers. Due to limitations in the software, individually lowering those numbers now would require manually editing each request, as would granting only some of the requests at once rather than the whole list.

    So, two decisions were made:

    1) Everyone with a pending request will receive 1 invitation, just to clear out the backlog.

    2) User requests are being re-opened! You can now request a maximum number of 10 invitations at one time. Even with this hard limit in place, we ask that everyone ask for only what they need at a time. Once we've hit the figurative switch and re-enabled this feature later today you will be able to request invitations from your Invite a friend page.

    We very much appreciate all of our users, and we are proud of our growth this year, even through the bumpy times. We are glad that once again we can enable you to bring more people on board!

  • Archive maintenance today - downloads will be unavailable

    By .Lucy Pearson on Monday, 17 December 2012 - 11:52am
    Message type:
    Список тегов:

    The Archive of Our Own will be undergoing some maintenance today at approximately 18.00 UTC (what time is this in my timezone?). During the maintenance period, which will last approximately two hours, downloads will not work. You will still be able to browse and read on the Archive, but will not be able to download any works. If the work proves complicated, we may also have a period of downtime (although we hope to avoid this).

    What's going on?

    In the next few weeks, we'll be adding some new servers to the OTW server family. The new servers will add some extra capacity to the Archive of Our Own, and will also create extra room for Fanlore, which is growing rapidly thanks to the amazing work of thousands of fannish editors (as Fanlore users are well aware, this expansion has been putting the existing Fanlore server under increasing strain).

    In preparation for these new servers, we need to first reorganise the setup of the existing servers in order to free some more physical space at our colocation host without buying more rack space (rack space costs money, so it’s nice not to use more than we need). In order to do this, we’ll have to take some of the servers offline for a little while today. Doing this now will minimize the disruption caused when the servers arrive during the holiday period, which is typically one of the busiest times of year for the Archive.

    The Archive is set up so it can function without all servers running at once, so today, we will only have to take the server which hosts downloads offline. This means that attempts to download any work will fail while we reorganize our data, though the rest of the site will work as usual (pending any unexpected problems). If you prefer to read downloaded works, you may wish to stock up now! Downloads will be restored as soon as we finish our maintenance. We’ll keep you posted about further maintenance when the new servers arrive!

    Thanks for your patience while we do this work. You can keep track of current site status via our Twitter account AO3_Status.

  • Archive of Our Own Newsletter - November-December 2012

    By Camden on Thursday, 13 December 2012 - 8:00pm
    Message type:

    Hello and Happy Holidays from the AO3! This newsletter contains all sorts of fun facts and news from the past two months - read on for updates in the world of the Archive in November and December.

    Cool stuff on the Archive

    Filters are back and our Advanced Search form has been updated! Look at this post for details on how to use our new filters and Advanced Search form. Release 0.9.2 went live and included more than just tag filters; we were also able to deploy bookmark filtering, a new search form for works and bookmarks, and new index pages for works and bookmarks. Releases 0.9.3 and 0.9.3 Redux also went live and included a lot of bug squashing and fixes as well as the ability to anonymously comment on your own anonymous work.

    We're now issuing 750 user invites per day. Our queue is slowly but steadily decreasing. To those of you still on the waiting list, thank you for your continued patience!

    Your Archive needs you!

    The Content Policy Committee is working on changes to the AO3's Terms of Service and would like your feedback on the proposed changes. The content change files are available at the above news post for your consideration. We welcome all feedback!

    Bugs we’ve squashed

    We've been keeping a close eye on performance after the release of the new search engine and filters, and we're pleased with how it's holding up: it's a big improvement over the old system. We quickly located a bug causing tag wrangling changes not to propagate through to works and bookmarks properly, so tag wrangling was disabled while that was being fixed, and it's up and running again now.

    We also tackled a number of collection and challenge-related bugs - 36 of them in Release 0.9.3 alone! Special thanks to Scott in particular for doing a ton of work on those.

    What’s up in the world of tags?

    There's been a few changes in the Marvel Avengers fandoms (some of the Archive's most popular!) To prepare for the coming sequels, the Thor and Captain America movie fandom names have changed, to Thor (Movies) and Captain America (Movies).

    Also, following the official material, the main metatag for The Avengers (2012) and the related movies is now Marvel Cinematic Universe.

    Note that if you tag for any of the subfandoms (e.g. if you tag your work The Avengers (2012) or Iron Man (Movies) that the work will automatically appear under Marvel Cinematic Universe. You can also tag a work with MCU if you like, but it will filter the exact same way as a work tagged with only the subfandoms.

    You may also notice a tag The Avengers - Ambiguous Fandom. This ambiguous tag is necessary due to the existence of two unrelated "Avengers" fandoms, the Marvel superhero team and the British superspy series, and includes all works from both these series. As with MCU, using this tag in conjunction with more specific tags won't change where your work appears (and tagging with only this tag will make your Avengers movie-verse fic less likely for other users to find.) Generally, to make your works easier to find, tag with the most specific fandom (or fandoms) that apply.

    For more information on the wild and wacky world of Marvel Avengers tags, please check out this informative post by one of our brave Avengers wranglers!

    Adventures with Support

    We're still keeping up with support tickets, even in spite of the start of holiday absences. In fact, we're about to hit our 10,000th ticket in our tracking program - that's just about 3,300 tickets a year! Every ticket sent in with an email address gets an individual response, and even the ones without email are still documented as appropriate with other committees. We're incredibly proud of our Support teams over the years and all they've done for the Archive!

    AD&T Committee business of note

    Thank you to all the coders, testers, and volunteers who helped us with Release 0.9.2 and 0.9.3! There's been a ton of work going on behind the scenes, and we're enormously grateful to everyone who's been helping out, and to the tag wranglers for their patience while we've sorted out some bugs.

    We've been cautiously optimistic about performance over the last month, and we've ordered three new servers that we're hoping will help us to keep things running smoothly as we head into 2013. That's enabled us to increase the number of invitations that we send out each day, which is helping to cut down the wait times significantly.

    We're working on one last release for 2012, which will be a small one, focused on bug fixes. There's already code in the pipeline for the first release of 2013, and we're looking forward to adding some long-awaited improvements and new features in the new year!

    Tag Wrangling Committee business of note

    Due to changes in the search engine with the main new filters deploy, tag wrangling was turned off for most of November. With the latest deploy, wrangling is working again and our wranglers have been busy getting all the new tags of the last month in order (our amazing volunteers managed to wrangle over 300 uncategorized fandoms down to less than 5 in a single weekend!)

    Questions? Comments?

    We welcome feedback from users! If you have questions or comments, feel free to leave them in the comments of the latest news post, or send in a Support request (if you're reporting a bug, please send that to Support, as they're super efficient - comments on our news posts sometimes get overlooked).

    Mirrored from an original post on AO3 News. Find related news by viewing our tag cloud.


Subscribe to Archive Update