Random Server Crashes..
- pooty
- Posts: 4537
- Joined: Sat Apr 03, 2021 10:22 am
- Location: Michigan
- Server Sponsor: Yes
- Server Admin: Yes
Re: Random Server Crashes..
I am going to turn off the custom scoring for now and see if that fixes the issue...if it does we know where to go fix the crash with custom scoring.
I've got some fixes I think for custom scoring and the focus/crash issue, and it might be more than just turning it off but have to test them out a bit. Ultimately, I think we can get custom scoring working right and not crash, but if we can't I think it might be easier to have 2pt games, and either accept the occasionaly 1 round 2pt match, or set the OT Time to like 2 minutes, and reduce core drain speed. So pretty much all matches are OT 1pt rounds.
I've got some fixes I think for custom scoring and the focus/crash issue, and it might be more than just turning it off but have to test them out a bit. Ultimately, I think we can get custom scoring working right and not crash, but if we can't I think it might be easier to have 2pt games, and either accept the occasionaly 1 round 2pt match, or set the OT Time to like 2 minutes, and reduce core drain speed. So pretty much all matches are OT 1pt rounds.
- captainsnarf
- Posts: 2714
- Joined: Tue Jul 06, 2021 1:51 pm
- Location: Washington
- Server Sponsor: Yes
- Server Admin: Yes
- Contact:
Re: Random Server Crashes..
Yes I think this is the way to go. Commenting out a couple lines here and there is too slow in isolating the problem. Whole sections need to be removed until it's found. Divide and conquer.
It could end up being something fundamental with the engine about how game rules are handled. It could be something really dumb. Can you spot the bug?
Code: Select all
function Destroyed()
{
if(Trail != None)
Trail.Destroy();
super.Destroy();
}
Thank you for looking into this. I've been pretty busy lately.
- pooty
- Posts: 4537
- Joined: Sat Apr 03, 2021 10:22 am
- Location: Michigan
- Server Sponsor: Yes
- Server Admin: Yes
Re: Random Server Crashes..
Yes, that's what I am thinking, there's something in Rules recursion/calling that either isn't returning properly or is not ending/destroying in the right order. Or calling Destroy on something thats already Destroyed....It could end up being something fundamental with the engine about how game rules are handled. It could be something really dumb. Can you spot the bug?
I think the key will be to get the round to end "normally" to set the focus and what not without any special work and slowly add in back from there.
- pooty
- Posts: 4537
- Joined: Sat Apr 03, 2021 10:22 am
- Location: Michigan
- Server Sponsor: Yes
- Server Admin: Yes
Re: Random Server Crashes..
So one thing I've narrowed down, working on EvenMatch custom score.
Level.GRI.Teams[Scorer.Team.TeamIndex].Score -= oldscore;
Level.GRI.Teams[Scorer.Team.TeamIndex].Score += newscore;
if(Level.GRI.Teams[Scorer.Team.TeamIndex].Score < 0)
Level.GRI.Teams[Scorer.Team.TeamIndex].Score = 0;
Screws with the timer. Meaning if you kill the core in regulation, it starts the next round timer with the remaining round time from the first round. When that runs out it doesn't go into OT. This looks like at least to me something we need to fix. That right there tells me some where it isn't getting reset.... almost like score changes, something fires elsewhere, and then we change the real score.. and something fubars.
Good news is other than that, I seem to have it working with much less code. I am testing completely without UTComp for now, locally and haven't had the need for the additional SetFocus stuff.
I am almost wondering if we just pull custom scoring out of EvenMatch. I think for now that should help with these weird crash issues.
I think we implement the custom scoring as seperate mutator, and I think we can do it easier:
On ONSOnslaughtGame:
MainCoreDestroyed Function is, where scoring is hard coded (gasp!).
Think its possible to make a mutator that basically says (in PostNetBegin play?)
We should ideally only need to replace that function and two lines at that.
But bottom line I think I am going to rebuild EvenMatch for now without any custom scoring. That seems to be problematic for now in my local testing. And I'm not sure that EvenMatch is the right spot anyway, I think we just put it there since we're customizing it anyway, but custom scoring really isn't part of team balancing.
Level.GRI.Teams[Scorer.Team.TeamIndex].Score -= oldscore;
Level.GRI.Teams[Scorer.Team.TeamIndex].Score += newscore;
if(Level.GRI.Teams[Scorer.Team.TeamIndex].Score < 0)
Level.GRI.Teams[Scorer.Team.TeamIndex].Score = 0;
Screws with the timer. Meaning if you kill the core in regulation, it starts the next round timer with the remaining round time from the first round. When that runs out it doesn't go into OT. This looks like at least to me something we need to fix. That right there tells me some where it isn't getting reset.... almost like score changes, something fires elsewhere, and then we change the real score.. and something fubars.
Good news is other than that, I seem to have it working with much less code. I am testing completely without UTComp for now, locally and haven't had the need for the additional SetFocus stuff.
I am almost wondering if we just pull custom scoring out of EvenMatch. I think for now that should help with these weird crash issues.
I think we implement the custom scoring as seperate mutator, and I think we can do it easier:
On ONSOnslaughtGame:
Code: Select all
event InitGame(string Options, out string Error)
{
// Register all the PowerCores
if (N.IsA('ONSPowerCore'))
{
PowerCores[x] = ONSPowerCore(N);
PowerCores[x].NotifyUpdateLinks = UpdateLinks;
PowerCores[x].OnCoreDestroyed = MainCoreDestroyed;
Code: Select all
function MainCoreDestroyed(byte T)
{
local Controller C;
local PlayerController PC;
local int Score;
if (bOverTime)
Score = 1;
else
Score = 2;
if (T == 1)
{
BroadcastLocalizedMessage( class'ONSOnslaughtMessage', 0);
TeamScoreEvent(0, Score, "enemy_core_destroyed");
Teams[0].Score += Score;
Teams[0].NetUpdateTime = Level.TimeSeconds - 1;
CheckScore(PowerCores[FinalCore[1]].LastDamagedBy);
}
else
{
BroadcastLocalizedMessage( class'ONSOnslaughtMessage', 1);
TeamScoreEvent(1, Score, "enemy_core_destroyed");
Teams[1].Score += Score;
Teams[1].NetUpdateTime = Level.TimeSeconds - 1;
CheckScore(PowerCores[FinalCore[0]].LastDamagedBy);
}
//round has ended
for (C = Level.ControllerList; C != None; C = C.NextController)
{
PC = PlayerController(C);
if (PC != None)
{
PC.ClientSetBehindView(true);
PC.ClientSetViewTarget(PowerCores[FinalCore[T]]);
PC.SetViewTarget(PowerCores[FinalCore[T]]);
if (!bGameEnded)
PC.ClientRoundEnded();
}
if (!bGameEnded)
C.RoundHasEnded();
}
ResetCountDown = ResetTimeDelay;
}
Code: Select all
x = 0;
for(N = Level.NavigationPointList; N != None; N = N.NextNavigationPoint)
{
// Register all the PowerCores
if (N.IsA('ONSPowerCore'))
{
PowerCores[x].OnCoreDestroyed = MainCoreDestroyed_withOmniCustomScoring;
if (PowerCores[x].bFinalCore)
FinalCore[PowerCores[x].DefenderTeamIndex] = x;
x++;
}
But bottom line I think I am going to rebuild EvenMatch for now without any custom scoring. That seems to be problematic for now in my local testing. And I'm not sure that EvenMatch is the right spot anyway, I think we just put it there since we're customizing it anyway, but custom scoring really isn't part of team balancing.
- pooty
- Posts: 4537
- Joined: Sat Apr 03, 2021 10:22 am
- Location: Michigan
- Server Sponsor: Yes
- Server Admin: Yes
Re: Random Server Crashes..
I have a seperate mutator that replaces the MainCoreDestroyed:
Scoring works as expected. MainCoreDestroyedOmniScoring is pretty much cut/copy from stock UT MainCoreDestroyed except the hard coded scores...
But Still have the clock/timer issue though... just like the other one. WTF?
However this is IMO, much cleaner for custom scoring. I'll play with it a bit more.
As far as the server, Custom Scoring is turned off, which should stop the crashes at the expense you can now have 2pt matches...I'll work on removing the custom scoring stuff from EvenMatch.
Code: Select all
function PostBeginPlay()
{
log(Class$" build "$Build, 'OmniONSScoring');
ONSGame = ONSOnslaughtGame(Level.Game);
}
function bool CheckReplacement(Actor Other, out byte bSuperRelevant)
{
local ONSPowerCore PC;
PC = ONSPowerCore(Other);
if (PC!=None) {
PC.OnCoreDestroyed = MainCoreDestroyedOmniScoring;
if (bDebug) log("Setting OnCoreDestoryed = Omni",'OmniONSScoring');
}
return true;
}
Scoring works as expected. MainCoreDestroyedOmniScoring is pretty much cut/copy from stock UT MainCoreDestroyed except the hard coded scores...
But Still have the clock/timer issue though... just like the other one. WTF?
However this is IMO, much cleaner for custom scoring. I'll play with it a bit more.
As far as the server, Custom Scoring is turned off, which should stop the crashes at the expense you can now have 2pt matches...I'll work on removing the custom scoring stuff from EvenMatch.
- pooty
- Posts: 4537
- Joined: Sat Apr 03, 2021 10:22 am
- Location: Michigan
- Server Sponsor: Yes
- Server Admin: Yes
Re: Random Server Crashes..
Cleaned up version of UTCompOmni and EvenMatchOmni .. without any Custom Scoring code are on the server.
I left times same, so you can get 2pt victory if you kill the core < 4mins. So its kind of like a mulligan but you get a new map.
I'll work on the custom scoring, but I have a feeling the custom scoring was causing crashes, because the end of round timer wasn't getting set right ever....so there's some event somewhere not getting called or as Snarf mentioned some kind of Timer (it thinks round is still going) issue.
Hopefully this stabilizes things until I fix the custom scoring....its possible it might be a ton of work, in which case I suggest we just play with round time/OT time/core drain.
And slight off topic. I see that now I can replace functions (not just values) on any class. I suppose its not surprising since under the hood its C++, but still handy thing to have in the toolbox.
I left times same, so you can get 2pt victory if you kill the core < 4mins. So its kind of like a mulligan but you get a new map.
I'll work on the custom scoring, but I have a feeling the custom scoring was causing crashes, because the end of round timer wasn't getting set right ever....so there's some event somewhere not getting called or as Snarf mentioned some kind of Timer (it thinks round is still going) issue.
Hopefully this stabilizes things until I fix the custom scoring....its possible it might be a ton of work, in which case I suggest we just play with round time/OT time/core drain.
And slight off topic. I see that now I can replace functions (not just values) on any class. I suppose its not surprising since under the hood its C++, but still handy thing to have in the toolbox.
- pooty
- Posts: 4537
- Joined: Sat Apr 03, 2021 10:22 am
- Location: Michigan
- Server Sponsor: Yes
- Server Admin: Yes
Re: Random Server Crashes..
So no crashes but we got a few rounds with the funky running around and not getting the focus set. Which makes no sense, since the round stuff is now all stock... There's nothing to interfere with it..unless there's something in UTComp that messes with it unintentionally?
I mean even Team DM matches change the focus at the end and UTComp worked with those right?
I mean even Team DM matches change the focus at the end and UTComp worked with those right?
- pooty
- Posts: 4537
- Joined: Sat Apr 03, 2021 10:22 am
- Location: Michigan
- Server Sponsor: Yes
- Server Admin: Yes
Re: Random Server Crashes..
Snarf, any ideas on: few rounds with the funky running around and not getting the focus set. ? I can't duplicate it on my testing. I run dedicated server and other client. Running UTCompOmni, EvenMatchOmni on my dedicated host everything works perfectly. Now running UT2004 client (64-bit) so its a completely seperate process from the server.
- captainsnarf
- Posts: 2714
- Joined: Tue Jul 06, 2021 1:51 pm
- Location: Washington
- Server Sponsor: Yes
- Server Admin: Yes
- Contact:
Re: Random Server Crashes..
Have you pushed the latest to github? I'll check it out when I get off work.
'MainCoreDestroyed' in ONSOnslaughtGame is what stops the running around thing.
Before it stops the running around, it does a call to CheckScore. This was where the custom scoring was hooked into. I was careful to return whatever true/false value that super.CheckScore() did, but I think that might be wrong. I think we *always* want to return false.
Another issue is inside CheckScore(), there is a GotoState('RoundEnded') or something like that. I think that transition may not be happening sometimes, or it does happen which causes it to skip the remaining code in MainCoreDestroyed method.
Either way, all of the issue is inside that MainCoreDestroyed method.
'MainCoreDestroyed' in ONSOnslaughtGame is what stops the running around thing.
Before it stops the running around, it does a call to CheckScore. This was where the custom scoring was hooked into. I was careful to return whatever true/false value that super.CheckScore() did, but I think that might be wrong. I think we *always* want to return false.
Another issue is inside CheckScore(), there is a GotoState('RoundEnded') or something like that. I think that transition may not be happening sometimes, or it does happen which causes it to skip the remaining code in MainCoreDestroyed method.
Either way, all of the issue is inside that MainCoreDestroyed method.
- pooty
- Posts: 4537
- Joined: Sat Apr 03, 2021 10:22 am
- Location: Michigan
- Server Sponsor: Yes
- Server Admin: Yes
Re: Random Server Crashes..
I'll load up the CustomScoring mutator I wrote into Git.
It just replaces the MainCoreDestroyed function which is identical except for the score values, that is its a direct copy from base onslaught code (some changes for scope). It works for scoring. But when a round ends early function executes, gives 1 pt instead of two, but the base UT timer isn't reset. I suspect there's some code elsewhere that handles that reset. I am thinking of just resetting it at the end of main core destroyed in my mutator...
But my original question was even without any custom scoring, using the stock code right now on the server, without any custom scoring, we sometimes don't get end of round setfocus stuff done.. which is strange because its all now base Onslaught code. Which I can't duplicate locally.
It just replaces the MainCoreDestroyed function which is identical except for the score values, that is its a direct copy from base onslaught code (some changes for scope). It works for scoring. But when a round ends early function executes, gives 1 pt instead of two, but the base UT timer isn't reset. I suspect there's some code elsewhere that handles that reset. I am thinking of just resetting it at the end of main core destroyed in my mutator...
But my original question was even without any custom scoring, using the stock code right now on the server, without any custom scoring, we sometimes don't get end of round setfocus stuff done.. which is strange because its all now base Onslaught code. Which I can't duplicate locally.
Yes there's some timing off... somewhere. I think I could use my Mutator to log what happens where, now that we can change 'MainCoreDestroyed'Another issue is inside CheckScore(), there is a GotoState('RoundEnded') or something like that. I think that transition may not be happening sometimes, or it does happen which causes it to skip the remaining code in MainCoreDestroyed method.