wes

Attention XBOX ONE Players: Stability and Retro Jason

72 posts in this topic

I wanted to take a moment to update our Xbox One Users. Both on the current state of the build and the imminent “Retro Jason” patch.
 
Retro Jason Patch
 
Unfortunately, our patch that was to include the Retro Jason has not passed certification for Xbox One. This was due in part to some instability of the new content AND the memory leak issue mentioned below. This is an unfortunate turn of events, as we had hoped to add the skin and content while continuing to work on these issues, however, this has become our top priority for Xbox One users. We’ve narrowed some memory leak crashes down, and we are currently back in certification and are hoping to have the new content out to you just as soon as we can, however, we are not certain when that will be.
 
Crashes
 
The XB1 build of the game has been unstable since launch. I know, I know..tell you something you don’t know, right? The problem is a memory leak in the engine, and our guys are working to fix this leak. I know it appears on the surface that PC and PS4 players are getting some favoritism on this front, but that’s not true. We have been working very hard on the XB1 version, this has just been a tougher problem to track down. When we test internally, we can play the game for 15-20 matches in a row on XB1 and only have one player crash out. On the exact same build, we can test again and get 3-4 players crashing out after only a few matches. Again, this all points to a memory leak. 
 
So what is a memory leak?
 
Basically, when a program (in our case a game) uses memory, it is supposed to “release” said memory when not in use. When a leak occurs, memory is not being released properly or in some cases at all. Almost every game suffers from this, but they are usually so small, that the end user doesn’t notice it. Ours is very noticeable. We’ve narrowed some down and have fixed them but there are more. We are actively working on finding and fixing them as fast as we can. I wish we knew the exact amount of time required to fix it, but at times can be like a needle in a haystack. But we’re getting some outside help, from other colleagues in the industry that have had this identical problem. This in an effort to speed it up, or at least avoid some hurdles getting to the root of the problem. 
 
What we Know
 
Here’s what we do know. We failed certification, we then immediately put in some quick fixes and re-submitted for certification with MS early this morning. Tomorrow morning (Wednesday) we will get an update from MS on this patch. If everything checks out and we pass, then Retro Jason will be available within that patch. This could be same day or Thursday morning. However, if we fail certification again, we will continue to work on fixes to get this content out ASAP.   
 
I want to personally apologize to all XB players. Both for the memory leak causing all the crashes, for the bad matchmaking, and for this content delay. We were very excited to get XB players this awesome update, as you all deserve some cool stuff, given the circumstances. This is on me and I’m sorry. We’re doing our best to make it right, just as fast as we can. As soon as I know more, you will know more. You are our priority right now, and few of us are sleeping until we solve it. 
 
-W  
 
 

2 people like this

Share this post


Link to post
Share on other sites
28 minutes ago, wes said:

Words

Appreciate the update

If the patch isn't in by the 23rd will we still receive the double xp weekend? 

Share this post


Link to post
Share on other sites

They knew this morning it wasn't coming out, and dragged it out all day to tell us, That's the most damning part to me, have some respect for us and dont keep us waiting all day for something you know isn't coming out, total chad move.

Share this post


Link to post
Share on other sites

Im so done with my xbox. After todays news i just want to throw it away. I wish i had purchased this game on my ps4 so i wouldnt feel like i wasted the 140 hours i have on this game. Dont get me wrong, i love this game. It has just opened my eyes up to how much i dont like xbox anymore lol  

Share this post


Link to post
Share on other sites
35 minutes ago, wes said:

I wanted to take a moment to update our Xbox One Users. Both on the current state of the build and the imminent “Retro Jason” patch.
 
Retro Jason Patch
 
Unfortunately, our patch that was to include the Retro Jason has not passed certification for Xbox One. This was due in part to some instability of the new content AND the memory leak issue mentioned below. This is an unfortunate turn of events, as we had hoped to add the skin and content while continuing to work on these issues, however, this has become our top priority for Xbox One users. We’ve narrowed some memory leak crashes down, and we are currently back in certification and are hoping to have the new content out to you just as soon as we can, however, we are not certain when that will be.
 
Crashes
 
The XB1 build of the game has been unstable since launch. I know, I know..tell you something you don’t know, right? The problem is a memory leak in the engine, and our guys are working to fix this leak. I know it appears on the surface that PC and PS4 players are getting some favoritism on this front, but that’s not true. We have been working very hard on the XB1 version, this has just been a tougher problem to track down. When we test internally, we can play the game for 15-20 matches in a row on XB1 and only have one player crash out. On the exact same build, we can test again and get 3-4 players crashing out after only a few matches. Again, this all points to a memory leak. 
 
So what is a memory leak?
 
Basically, when a program (in our case a game) uses memory, it is supposed to “release” said memory when not in use. When a leak occurs, memory is not being released properly or in some cases at all. Almost every game suffers from this, but they are usually so small, that the end user doesn’t notice it. Ours is very noticeable. We’ve narrowed some down and have fixed them but there are more. We are actively working on finding and fixing them as fast as we can. I wish we knew the exact amount of time required to fix it, but at times can be like a needle in a haystack. But we’re getting some outside help, from other colleagues in the industry that have had this identical problem. This in an effort to speed it up, or at least avoid some hurdles getting to the root of the problem. 
 
What we Know
 
Here’s what we do know. We failed certification, we then immediately put in some quick fixes and re-submitted for certification with MS early this morning. Tomorrow morning (Wednesday) we will get an update from MS on this patch. If everything checks out and we pass, then Retro Jason will be available within that patch. This could be same day or Thursday morning. However, if we fail certification again, we will continue to work on fixes to get this content out ASAP.   
 
I want to personally apologize to all XB players. Both for the memory leak causing all the crashes, for the bad matchmaking, and for this content delay. We were very excited to get XB players this awesome update, as you all deserve some cool stuff, given the circumstances. This is on me and I’m sorry. We’re doing our best to make it right, just as fast as we can. As soon as I know more, you will know more. You are our priority right now, and few of us are sleeping until we solve it. 
 
-W  
 
 

I gotta ask and I doubt Ill get a real answer, but what was the in-house testing like on the xb1 prior to release? I mean all of these memory leaks had to be there in the builds you all were playing prior to release. I guess I just don't understand the whole process. Its almost like you knew you were releasing a lemon to us but didn't give us any heads up. 

 

Share this post


Link to post
Share on other sites

Double XP coming to Xbox users this weekend even if the patch isnt ready?

Share this post


Link to post
Share on other sites
38 minutes ago, 2moar said:

They knew this morning it wasn't coming out, and dragged it out all day to tell us, That's the most damning part to me, have some respect for us and dont keep us waiting all day for something you know isn't coming out, total chad move.

We didn't know this morning. We submitted to MS and there was a chance of same day turn around. But that did not occur. I waited until MS informed me that they needed another day to test.

Share this post


Link to post
Share on other sites

Appreciate the info Wes, and I don't believe for one minute that you guys are trying to screw over one group of players, or favor one group over another. However, these issues are tiresome. Perhaps the certification could have been tested, and passed before an announcement was made about the content? And the crashing...I'm sure there's no need for me to go on a rant about the frustration that has caused. So I'll just end by saying I appreciate the free content (assuming I get it), and please do whatever possible to get the Xbox issues resolved. I know I'm not alone when I say patience is wearing thin.

2 people like this

Share this post


Link to post
Share on other sites
3 minutes ago, wes said:

We didn't know this morning. We submitted to MS and there was a chance of same day turn around. But that did not occur. I waited until MS informed me that they needed another day to test.

I think he is referring to the FB post where you all said it failed certification

Share this post


Link to post
Share on other sites
36 minutes ago, urbt said:

I gotta ask and I doubt Ill get a real answer, but what was the in-house testing like on the xb1 prior to release? I mean all of these memory leaks had to be there in the builds you all were playing prior to release. I guess I just don't understand the whole process. Its almost like you knew you were releasing a lemon to us but didn't give us any heads up. 

 

It's sporadic. And we also see differences on gen 1 XB vs XB1 S. Also, when testing internally we get "call stacks" or and error message the informs us as to where the problem is occurring. But with this crash, we don't get call stacks. When you send off to certification, you get a more thorough report from PS/MS so we can find issues. You also have a QA testing done, which we hired an outside company to help with. We saw the memory leaks, but again, our tests did not reveal the crashes happening all the time. Same from QA...same from Certification. If it passes cert, then it's good to go. Even if there's the occasional crash. But something was compounding the issue. Something that wasn't being caught in all tests. Our tests, our QA or cert. Not until just recently. We knew there was a memory leak, but weren't sure of the severity of it. After digging more and talking with other UE4 dev teams, they started sharing information with us. Where to look, and what to look for. 

1 person likes this

Share this post


Link to post
Share on other sites
3 minutes ago, Super Ty said:

Appreciate the info Wes, and I don't believe for one minute that you guys are trying to screw over one group of players, or favor one group over another. However, these issues are tiresome. Perhaps the certification could have been tested, and passed before an announcement was made about the content? And the crashing...I'm sure there's no need for me to go on a rant about the frustration that has caused. So I'll just end by saying I appreciate the free content (assuming I get it), and please do whatever possible to get the Xbox issues resolved. I know I'm not alone when I say patience is wearing thin.

Yes, I agree. The thing is...the build passed cert previously. We added in Retro Jason and then it failed. So we were kinda scratching our heads too. Which again leads to the sporadic nature of this crashing issue. But we at least have a strong lead now to the problem, thanks to some other dev teams that have gone down this path previously.

1 person likes this

Share this post


Link to post
Share on other sites
7 minutes ago, urbt said:

I think he is referring to the FB post where you all said it failed certification

Yes it failed and we learned of that last night. So we immediately resubmitted in hopes that it could be tested and certified today. However more time was needed. The moment I found that out, we put this post together to alert everyone.

Share this post


Link to post
Share on other sites
7 minutes ago, wes said:

It's sporadic. And we also see differences on gen 1 XB vs XB1 S. Also, when testing internally we get "call stacks" or and error message the informs us as to where the problem is occurring. But with this crash, we don't get call stacks. When you send off to certification, you get a more thorough report from PS/MS so we can find issues. You also have a QA testing done, which we hired an outside company to help with. We saw the memory leaks, but again, our tests did not reveal the crashes happening all the time. Same from QA...same from Certification. If it passes cert, then it's good to go. Even if there's the occasional crash. But something was compounding the issue. Something that wasn't being caught in all tests. Our tests, our QA or cert. Not until just recently. We knew there was a memory leak, but weren't sure of the severity of it. After digging more and talking with other UE4 dev teams, they started sharing information with us. Where to look, and what to look for. 

Alright well then you gotta see where my anger towards you guys is coming from with regards to the job that's being done. I will say that the last patch you guys did has greatly reduce the dashboard mem leaks for me. I think may 1 outta 20 or so games thus far. So that's great progess, I am just disappointed to read that you all were semi aware of these issues prior to release. 

Share this post


Link to post
Share on other sites

What a joke. Is it amateur hour over there?!!? You guys don't even know how to troubleshoot the game engine you are using to make your game? Talking with other UE4 dev teams? It sounds worse then you guys are letting on.

You should not have released the update if it was not ready to go on all platforms. Your PR is terrible. You've further alienated the Xbox community more than you already have. Gun Media and Illfonic should be ashamed of themselves. 

Share this post


Link to post
Share on other sites
9 minutes ago, wes said:

Yes it failed and we learned of that last night. So we immediately resubmitted in hopes that it could be tested and certified today. However more time was needed. The moment I found that out, we put this post together to alert everyone.

Since you are here, do you think you could take a moment out of your day to comment on whether we will ever be getting the dedicated servers that were promised? It would certainly be appreciated.

1 person likes this

Share this post


Link to post
Share on other sites
1 hour ago, Kato said:

Im so done with my xbox. After todays news i just want to throw it away. I wish i had purchased this game on my ps4 so i wouldnt feel like i wasted the 140 hours i have on this game. Dont get me wrong, i love this game. It has just opened my eyes up to how much i dont like xbox anymore lol  

Why are you blaming Xbox? They aren't at fault....

Its the amateur dev team releasing a broken update for a game that is still broken. 

Can we expect this for every update/DLC for Xbox in the future?

Share this post


Link to post
Share on other sites

Really feel for the XB1'ers. Hopefully the single player patch doesn't completely explode everyone's console.

Share this post


Link to post
Share on other sites

Sooo you guys were aware of the crashing on Xbox and didn't tell us. Were you guys also aware of the matchmaking issues?

Im not even going to pretend that I understand game designing, but this is starting to get ridiculous. My game crashes once every 5/6 games. No telling how much exp/cp I've lost do to the crashing. Then on top of it we don't get the update, and are literally told at the last minute it's been delayed. 

JUST like the last patch Xbox got. Delayed for 5 days. So I guess that's when we can expect to get this update.

Will we at least get the double exp this weekend even if we don't get the patch, or are we just gonna be SOL there too?

Oh and please give us an answer about dedicated servers on consoles, you got people on here that would love an actually response on that particular matter.

1 person likes this

Share this post


Link to post
Share on other sites
1 hour ago, urbt said:

Alright well then you gotta see where my anger towards you guys is coming from with regards to the job that's being done. I will say that the last patch you guys did has greatly reduce the dashboard mem leaks for me. I think may 1 outta 20 or so games thus far. So that's great progess, I am just disappointed to read that you all were semi aware of these issues prior to release. 

All games launch with memory leaks. It's a very, very common problem. It's the severity that is the issue. And we were unaware of said severity until more recently. Kind of like peeling layers of an onion. We had a leak pertinent to animation, we found that, fixed it and that helped to stabilize most users. But then it came back. But it's hit or miss, which is why we struggled so much to find the issue with it. 

Share this post


Link to post
Share on other sites
1 hour ago, Stranglemania said:

What a joke. Is it amateur hour over there?!!? You guys don't even know how to troubleshoot the game engine you are using to make your game? Talking with other UE4 dev teams? It sounds worse then you guys are letting on.

You should not have released the update if it was not ready to go on all platforms. Your PR is terrible. You've further alienated the Xbox community more than you already have. Gun Media and Illfonic should be ashamed of themselves. 

Are you trying to suggest that dev teams on other games, don't reach out to other dev teams to ask questions? All game teams do this. They even bring in members of other teams to work on their games to help fight fires. It's understandable that you aren't aware that this is a very common occurrence in the gaming industry. But that's why we talk to you guys, to make you more informed about how games are made.

But again, the update was tested internally, then by a third party and then certified by MS/PS. But there are always issues with online games, it's just the nature of the beast. There also aren't a ton of UE4 games on the market, especially online ones. So there wasn't a massive database of FAQ's for us to work with. I understand your frustration though, it does suck. We have egg on our face, that is for sure. There are issues that we dropped the ball on or lack experience in, while others are beyond our control. But we are doing our best.

1 person likes this

Share this post


Link to post
Share on other sites
1 hour ago, RKSDooM said:

Since you are here, do you think you could take a moment out of your day to comment on whether we will ever be getting the dedicated servers that were promised? It would certainly be appreciated.

We never promised dedicated for console, but it is something that we want to do. Usually dedicated is more often used for shooters, as in FPS games there's a need for accuracy/predictive algorithms. Being a 3rd person, non-competitive game, we didn't feel there would need to be this support for console. But we want to implement a system so that if the host quits, the match would continue...and the best way to do that is with dedicated servers. So it's on the docket for sure.

1 person likes this

Share this post


Link to post
Share on other sites
1 hour ago, Bulldawg2010 said:

Sooo you guys were aware of the crashing on Xbox and didn't tell us. Were you guys also aware of the matchmaking issues?

Im not even going to pretend that I understand game designing, but this is starting to get ridiculous. My game crashes once every 5/6 games. No telling how much exp/cp I've lost do to the crashing. Then on top of it we don't get the update, and are literally told at the last minute it's been delayed. 

JUST like the last patch Xbox got. Delayed for 5 days. So I guess that's when we can expect to get this update.

Will we at least get the double exp this weekend even if we don't get the patch, or are we just gonna be SOL there too?

Oh and please give us an answer about dedicated servers on consoles, you got people on here that would love an actually response on that particular matter.

As mentioned before, all games crash, but it's the severity that is the issue. But we did discuss the crashing and match making issue a few times since launch..both here, on twitter, discord and on FB. As for when the patch comes out...we do know that MS has an updated patch in certification now and we should hear something tomorrow. But we don't control certification or how long that process goes. It can be a day, three days, 5 days, perhaps a little more. We hand them the game, and then we wait. 

As for getting double XP and the CP bonus..this patch has elements built into the HUD so that you actually can see that you're getting double XP. We can turn on the double XP on the back end, but most users won't really "see" the bonus unless they are really looking at their XP before a match starts. If you follow. Either way, if the patch isn't live by this weekend we will turn it on...there will just be a lot of folks that probably won't notice the bonus and assume it isn't working. Does that make sense?

1 person likes this

Share this post


Link to post
Share on other sites
18 minutes ago, wes said:

We never promised dedicated for console, but it is something that we want to do. Usually dedicated is more often used for shooters, as in FPS games there's a need for accuracy/predictive algorithms. Being a 3rd person, non-competitive game, we didn't feel there would need to be this support for console. But we want to implement a system so that if the host quits, the match would continue...and the best way to do that is with dedicated servers. So it's on the docket for sure.

Thank you very much for responding, but please check the @Friday13thGame twitter account's post for FRIDAY 26 Sep 2016 at 9:42 AM.

It does state that you will launch with dedicated servers.

Share this post


Link to post
Share on other sites
31 minutes ago, wes said:

All games launch with memory leaks. It's a very, very common problem. It's the severity that is the issue. And we were unaware of said severity until more recently. Kind of like peeling layers of an onion. We had a leak pertinent to animation, we found that, fixed it and that helped to stabilize most users. But then it came back. But it's hit or miss, which is why we struggled so much to find the issue with it. 

I still disagree with how this was handled but Ill let you get back to it, Thanks for coming in and not hiding out in Discord

Share this post


Link to post
Share on other sites
Just now, RKSDooM said:

Thank you very much for responding, but please check the @Friday13thGame twitter account's post for FRIDAY 26 Sep 2016 at 9:42 AM.

It does state that you will launch with dedicated servers.

Ah, yeah we launched with dedicated servers on PC because you kinda have to. Steam doesn't provide the type of multiplayer servers required...and how they use "lists" for P2P was not the best route. So we went with dedi's on PC. But again, we acknowledge the need for it on console and have it in our pipeline. Just a lot of fires at the moment to put out. 

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now