Site is down, serious issues

We’re having serious issues with our database servers

I’ll use this thread to post updates

Update 10:45 CEST:
Currently checking all database tables. This takes a while…

Update 11:47 CEST:
Database check completed. Rebuild of disk array 87%

Update 13:03 CEST:
Rebuild complete. Check consistency of drives 1%

Update 13:25 CEST:
Check consistency of drives 29%

Update 13:40 CEST:
Check consistency of drives 46%

1 Like

The site is back up for now. More work has to be done to restore redundancy, but I have to go to Amsterdam to do that (servers are located there).

2 Likes

It’s down again. Working on it…

Update 21:48 CEST:
Working on temporary restoring volumes until I can get to Amsterdam for a permanent fix

Update 23:48 CEST:
Site is back up. If you have any issues, report them here

Very minor issue: The total check count at the top of the page (next to your username) is no longer incrementing when you check a movie. I’ve tried this on a couple of films, but only in Chrome.

The count is correct upon page refresh, however.

For me it is incrementing when I check a movie. Have you tried an hard refresh (CTRL+F5)?

I did just try a hard refresh. I also closed the browser and reopened (FF - I was working in Chrome before). Still not incrementing.

Browsers up-to-date. Also tried a mobile browser (Chrome for Android), and it didn’t increment there, either.

It works for me in Firefox.

Just restarted the computer, and it’s still not incrementing on the page. As soon as I load another page, the count is accurate. And ICM is definitely tracking all the checks.

I guess I win the award for “just because something is slightly different doesn’t mean it’s a problem.” :smile:

I can’t log in with either Chrome or Firefox, v2 or v3.

edit: have since been able to login with IE on v2, but checks aren’t recorded. Can’t check movies in any browser Chrome, Firefox, or IE.

I’m not able to get past the login screen on v3 with any browser no matter how many times I try to login.

For fun, I’m going to delete cookies on IE and see what happens.

I tried logging in on a different browser. It gives an error the first time “Whoops!Your form session has expired. Please submit the form again.”, but works on the second try with no error. On Firefox.

I’ve cleared cookies on Firefox and rebooted the machine - can’t login to v2 or v3 with Firefox 55.0.3 no matter how many times I try.

I can’t check movies

I am experiencing the same issue as cabra, the movies I attemt to check are not registered. I have also unsuccessfully attempted to add movies to one of my created lists.

Unfortunately we’re once again having issues. I’m on it

Update 10:23 CEST:
We’re back up, but slow. Our main database servers are down (hardware issues) and I’ve reconfigured an application server to act as database server. Unfortunately, this server has a lot less memory and no SSD disks, which makes the site slow. This is all I can do for now, I’ll go to Amsterdam on Wednesday to fix it permanently.

4 Likes

Thank you, Marijn. It’s all right now for me. :thumbsup: