Page 1 of 1

Server Status - June 3rd, 2020

Posted: Thu Jun 04, 2020 12:12 am
by Tyclo
Earlier today, June 3rd 2020, Sentinels Republic went offline for an extended period of time. For the sake of transparency and clarity, I'd like to share what the issue was.

Notes:
Please restart your Launcher to receive a client patch. - Without this update, you many not be able to access shuttle terminals at some starports.

For the time being, the following planets have had most or all of their content disabled:
  • Chandrila
  • Coruscant
  • Moraband
  • Jakku
  • Mandalore
  • Taanab
  • Kaas
Explanation:
In Star Wars Galaxies, permanent world objects not owned by players are stored in a client file called a snapshot. Snapshots do not consist of player owned items, or creatures/NPCs or even some world objects, but most are stored in these snapshot files. Example of such objects are large and enterable structures such as starports, bunkers and medical centers. As well as small objects like public crafting stations, fountains, chairs, walls or decorative items in and around cities and POIs.
sie-lok-snapshot.jpg
For easy reference by the server, each of these objects has a unique object ID associated with them. These IDs are used for multiple difference scenarios, such as a player interacting with the object or a player entering a structure or other functions.
sie-oid.PNG
These object IDs are not just associated with snapshot objects, but also shared with objects spawned into the world from snapshot and player owned items, such as base turrets, shuttle terminals and much more.

Cause:
Now to today's issue. While the actual limitation of these object IDs is a very high number, all SWGEmu server's tend to experience issues if the number of object IDs begin to climb too high. This morning we exceeded that number and the server attempted to reference an object ID that was higher than it could access, thus the server went down. Upon rebooting, the server began using an object ID outside of the valid range and this caused an instability that prevented the server from loading.

Solution:
The immediate solution was to rearrange the object IDs in some of our snapshot files to a value lower than the failure point, which was a time consuming process but allowed the server to load without running out of object IDs.

The side effect of this is that structures in each of the snapshots that were rearranged no longer match the server side files for the encounters. If it was just the structure IDs, the fix would be easy, but on the server, each room (Cell) also has a unique ID that is based off the parent structure object.
sie-oid-cells.PNG
What that means is that every creature/NPC/object that spawns inside of a snapshot structure that was rearranged would not appear at the correct position or even at all. To fix this, each room ID will need to be updated to align with the new structure IDs, which will take some time.

Code: Select all

spawnMobile("lok", "mercenary_aggro", 300, 3.362, -12.000, 40.973, 2.360, 5126275)
spawnMobile("lok", "mercenary_aggro", 300, 9.515, -12.000, 46.971, -105.002, 5126275)
spawnMobile("lok", "mercenary_aggro", 300, -1.314, -12.000, 67.672, 63.717, 5126275)
spawnMobile("lok", "mercenary_aggro", 300, 3.700, -12.000, 67.231, 98.437, 5126275)
spawnMobile("lok", "mercenary_aggro", 300, -49.555, -20.000, 25.315, 47.572, 5126277)
spawnMobile("lok", "mercenary_aggro", 300, -76.635, -20.000, 24.636, -95.990, 5126277)
spawnMobile("lok", "mercenary_aggro", 300, -75.849, -20.000, 68.961, 130.427, 5126277)
5126275 and 5126277 are examples of cell IDs, which will need to be updated.

Current Status:
Instead of leaving the server offline, it was opted to disable any encounters where there were structure object ID conflicts.

In conclusion, many of the new planets will be missing encounters. Content will be added back in as soon as possible during regular maintenance periods and will be announced whenever possible. Please be patient while work done as process is very time consuming.

Re: Server Status - June 3rd, 2020

Posted: Thu Jun 04, 2020 12:34 am
by Elo
Y2.02K!

Thanks for the hard work, as always.

Re: Server Status - June 3rd, 2020

Posted: Thu Jun 04, 2020 12:52 am
by fergjeezy357
I appreciate the explanation and transparency. I wish game companies had this much integrity. Thank you all for your hard work.

Re: Server Status - June 3rd, 2020

Posted: Thu Jun 04, 2020 1:24 am
by Rebonik
I for one on this server am very happy with all the hard work and dedication the the dev's have put into this game that we all love to play. I appreciate the openness and clarity that troubles are explained to us and even though to me the above explanation was as clear as mud ( you could have put all the above info in another language and I would not have understood it). But I wanted to say thank you for developing such a good version of the game that fundamentally started live with issues and working out a lot of the kinks so we could play it.

Re: Server Status - June 3rd, 2020

Posted: Thu Jun 04, 2020 2:54 am
by Jeremiah87
Is that bucket still on Corellia is the real question Tylco...prof still looking for it

Re: Server Status - June 3rd, 2020

Posted: Thu Jun 04, 2020 3:25 am
by Flyingdark
Thank you for the transparency, and thank you for all the hard work! This is truly the best SWG server i have ever played on.

Re: Server Status - June 3rd, 2020

Posted: Thu Jun 04, 2020 3:43 am
by amnferguson1991
Thank you to all the devs that dedicated their spare time on such short notice, and for taking the time to give us full insight into the issues. This write up is greatly appreciated and very knowledgeable for those interested in the dev side of things too :geek:.

Thank you to Coz & the team of CSRs for keeping the Discord updated during the downtime.

And a special shout out to Whibbs for crashing the server because every time he connects he profusely increments the object ID count.

Re: Server Status - June 3rd, 2020

Posted: Thu Jun 04, 2020 6:56 am
by Xordium
Showing the big development houses how to do their job by actually communiicating. Impressed as usual!