 #...
·
·
5
likes
|
|---|
|
Hello, some of you might now that since the last couple of weeks I'm doing a full dive into performance optimizations to improve the speed of AstroBin. Two weeks ago AstroBin was bombarded by requests from a spambot crawler with servers in Hongkong, Singapore, and Frankfurt. They weren't actually attacking the website to do harm, but just crawling it overly aggressively for whatever reason. AstroBin was very slow for a day or two, and even offline from time to time. The silver lining of this event is that now we have some very nice tooling to inspect performance bottlenecks. I have always done performance analysis, but this gave me the opportunity to re-discover some tools that now make it easier to find issues. As a result, I found a bunch of low-hanging fruits and since a few days now AstroBin is being really fast (I also threw a bunch more servers at it while I was fighting fire, which today I took off again for cost management reasons). The most significant improvement has been on the search page, which now is literally 5x faster than before! Anyway, sorry about digressing. TL;DR: I wanted to ask you if wanted to test a change that I find very important: making the images' pages load equipment and acquisitions asynchronously, so that the page loads quickly, and by the time you've seen the image and scrolled down to the technical card, that will have loaded too without impacting the initial page load. To test this, please go to this website: https://beta.astrobin.com/account/login Sign in with your regular AstroBin credential and look at images' pages. Make sure the equipment and the acquisition details in the technical card always load, and that all the data is there. It's easier if you check your own images so you'll know if something is missing! If you want, you can compare with the regular AstroBin by using www instead of beta. The loading of the technical card of images is the only thing that has changed. Thank you in advance and please let me know if you find something amiss! Salvatore |
3.01
#...
·
|
|---|
|
Seems to function nicely to me, though the prior page loading is pretty quick as well on my end. There is a difference in the Acquisition details output. Dates format is different, concatenated text instead of date range on separate lines. Dates are not clickable, in non-beta it searches for images on that date I guess (never used this before). Several fields are not available in the beta site:
Examples I was looking at: https://beta.astrobin.com/6ipzy9/ https://beta.astrobin.com/p3b7pe/ EDIT: is the "technical cards" scope of this test just the "Equipment" section? Maybe the things I listed above are not in scope... |
#...
·
·
1
like
|
|---|
| Thanks for testing! Yes, those things are in scope. I’ll check and fix these issues tomorrow in the morning! |
2.41
#...
·
|
|---|
| Wow I feel that loads the image a LOT faster. |
5.76
#...
·
|
|---|
Eddie Pons: Interesting. I'd give that a strong "maybe". It's certainly not slower, but I don't see any real difference in image load time (and the first note seemed to indicate only the equipment area changed not the image load)? |
0.90
#...
·
|
|---|
Out here in Arizona, I would call it a dead heat  . The technical cards--and images--load fast in both environments. For some images the beta is a little faster, for some the production environment is a little faster, and in some they seem equal. In most cases, I had to go back and forth between the two environments and test repeatedly to see the difference. I like the move of Resolution, File Size and Data Source to the upper section of Acquisition Details. Best, Jeff |
5.77
#...
·
|
|---|
|
Hi Salvatore, Testing a number of image pages side-by-side, I honestly can't see much of a difference in load speeds - I have to call it a tie. Bearing in mind that over here in Australia, I'm a long way away from your servers, so I suspect that transmission time is the dominant factor, rather than server speed. I noticed that the format / layout of the beta page is slightly different to the 'standard' page :- a. date format is now YYYY-MM-DD - which I greatly prefer to DD-Month-Year b. the date is no longer clickable - not a significant loss IMHO c. RA/DEC centres, pixel scale etc are no longer displayed d. the 'more info' button is no longer displayed e. the 'Find images in the same area' button is no longer displayed - which IMHO is a loss. f. Resolution, file size, location & source are now displayed beneath Moon Phase - IMHO an improvement. I'm not sure how often b-e are used, but IMHO centre RA/Dec are very useful. I hope that helps  Gary |
1.43
#...
·
|
|---|
|
Definitely seems faster to me. --Steve |
#...
·
·
1
like
|
|---|
|
Hi guys, thanks for testing! The broken dates and missing plate-solving information are fixed now. Please have another go and see if something else is missing, if you like. I appreciate that! PS: make sure you don't accidentally end up on the live server (www instead of beta in the URL). This can happen if you click on notifications or take a trip to the equipment explorer. No need to test anything other than the image page of existing images. The beta server uses the same database as the real astrobin server, so if you save something IT'S SAVED FOR REAL. There's no need to comment on speed, as there are other factors involved: 1. You're testing an environment with one server but only a handful of people using it. AstroBin normally uses 4 servers like that and it serves hundreds of people concurrently. Of course the beta server will be overall faster 2. Many things are cached on AstroBin so they don't hit the database or other servers unnecessarily, so especially if you open the same page repeatedly, you might just get a locally cached view 3. Other matters of concurrency and contention of resource might mean tens of a second here and there, practically randomly, anyway. Loading the equipment and acquisition details asynchronously AFTER the page has been rendered will significantly improve the speed at which the page initially loads and shows you the rest of the information. Loading the equipment, especially, is the slow part, and that's going to cut 150-200 ms off every page load. In the best scenario, this doesn't matter much: the fastest 50% of requests of image pages take less than 400ms, and it will go down to around 200ms, which is absolutely fantastic. The fastest 75% take 800ms or less, and 200ms off will still be noticeable. The fastest 95% take 1.1s or less, which means that the slowest 5% take over 1.1s, which is not great. But the cool thing is that WHEN these slow requests happen, it's usually due to waiting for the database, so I think that while that 5% will see the loading indicator for the equipment, at least the perceived speed will be much higher because the page will load fast, and they will anyway take some time to scroll down to the technical card question. This is a huge win! I'll have to do the same for the front page (the activity stream has the same issue: it's the bottleneck for that page, so I can begin loading it asynchronously). |
|
3.01
#...
·
|
|---|
| The updates look great to me and I see the image load before the equipment. Order of things loading is different and expected, I see the image load before equipment is fleshed out in the beta site. It will be nice to have that improvement, see the image pop up before technical details. Thanks! |
#...
·
·
1
like
|
|---|
This is now live! Thank you all for testing and reporting the issues. It looks like it's working really well, the image pages (most visited pages on AstroBin, obviously) are literally twice as fast on average! |
9.82
#...
·
|
|---|
| Salvatore, I hate to say it, but I'm still getting the "Oops Astrobin is momentarily unavailable", even just now. Tried updating some settings for one of my images after it was glitched (plate solving overlay would NOT turn off no matter what the setting) and it timed out after about 30 seconds. Don't know if this is related to your changes, but AB still feels very clunky during peak hours. |
#...
·
|
|---|
|
Hi @Brian Puhl, while my metrics do suggest really strong improvements since I landed all these performance improvements, there are still outlier requests that I need to figure out. The first step would be to isolate and measure these things. I've just set up a way to associate timings of HTTP requests and the relative drilldown of functions of the site that contribute to the timings to a user ID, so I can begin observing what goes on with your accesses specifically. However, I can only trace 1 in every 5 requests, therefore I would ask you the following: 1. Log out of AstroBin, and begin browsing the site anonymously. Click on images, forums, user profiles. You can't go to the home page because when you're logged out it will redirect you to the marketing site welcome.astrobin.com. Please do this for a while and then let me know if you feel that's slow too. 2. Log back in and start clicking your heart out. Try to make as many requests as possible. It would be great if you could set 10 minutes aside and just click on a lot of images, user profiles, forum posts, full-size images, Like images, etc. The more things you click (that navigate to a different page, especially) the more data I will have to see if there's a pattern of something that makes things slow specifically for you. If during these tests you don't find that the website is slow, then I don't think we have a problem. If you observe that pages like image pages, home page, user galleries, take longer than 1-2 seconds to load and this is very often, then there must be something wrong. If this happens only occasionally, i.e. a page taking longer than 1-2 seconds only once every 10 minutes, that's probably expected (find me a website that's faster than that consistently...) If you get "oops AstroBin is momentarily unavailable", now that's weird. It means that your request could not be routed for some reason (stuck on a server that was being replaced by an autoscaling operation? but it shouldn't happen) or that it took more than 60 seconds, which is also extremely weird. If that happens, please take note of the exact time (down to a few seconds if you can). Please get back to me with any observations using the Contact menu -> Support link above. Thank you! |
