Savegame Troubles

[nightflyer]

I recently made a move from WinXP to Win7. r3189 worked well back with XP and works on Win7 just as well.
Since I have a x64 computer now, I tried the latest x64 revsion(r3315).
Now whenever I load up a game(VC2[ULUS10515] and VC3[NPJH50343]), I get notified that the savedata is corrupt(the same savedata works fine for r3189).
Then I tried with the 32-bit r3315 => same result.
After that, I tried to let the game create a new savegame, but even that new savegame was considered to be corrupted when trying to load. And everytime I let the game delete the newly created corrupted savegame, the emulator goes into endlessly trying to delete something(Savedata MODE_SINGLEDELETE directory not found) after deleting all the files that were created before.

So neither the 32-bit nor the 64-bit r3315 can load my saves from r3189(they were created and used without the crypto-mode) and they can't create new(non-corrupt) savegames as well(the folder and files are created though). Changing to cryto-mode leads to the exact same result.
I don't think the problem is file-system related, since the DLC-data seems to load just fine.
Anyone got an idea what might cause this/how to fix this?

EDIT: r3304 seems to bee the last revision where loading my savegames doesn't result in the claim of "corrupted data"(i guess r3305 would still work if not for the incomplete build). I have he suspicion that the changes introduced in r3306 are the cause for this.

EDIT2: I've confirmend my suspicion. After a rollback to the last version for all the files modified by r3306 loading and saving work again. I think I'll try putting stuff back in litte by little and see what happens.

EDIT4: I now know that whatever breaks loading/saving is in HLE/modules150/sceUtility.java. I'll try modifing that code to make it work.

EDIT5: I think I have the problem: The savefiles I have are secure files[if (saveFileSecureEntriesAddr != 0) results in true]. Furthermore if(saveFileEntriesAddr != 0) results in a false-value. That alone would be no problem but with the new implementation, there is a if(savedataParams.isSecureFile(entry)) check before checking if we have a value for saveFileSecureEntriesAddr. And that check returns a false.
So I took a closer look to the isSecureFile() function. Said function accesses this value: psf.get("SAVEDATA_FILE_LIST"), which is not set for my savefiles.

What exactly is this SAVEDATA_FILE_LIST and why is it suddenly needed, even though things seem to work without it(at least for my saves)?

EDIT6: I forgot to mention, I had to change mem.write64(entryAddr + 8, ((stat.size + 0xF) & ~0xF) - 0x10); back to mem.write64(entryAddr + 8, stat.size); for things to work. What was that change supposed to do?

So now that I found a way to make saving/loading work for me again, could you explain what SAVEDATA_FILE_LIST and ((stat.size + 0xF) & ~0xF) - 0x10) are supposed to be doing gid15? Any idea why that changes made it impossible to load savegames for VC2[ULUS10515] and VC3[NPJH50343]?

[sum2012]

r3306 fix "load Motto Nuga Cel -JPN-ULJM05654
's save from crypto mode "

(07-19-2013 09:30 AM)nightflyer Wrote:  I recently made a move from WinXP to Win7. r3189 worked well back with XP and works on Win7 just as well.
Since I have a x64 computer now, I tried the latest x64 revsion(r3315).
Now whenever I load up a game(VC2[ULUS10515] and VC3[NPJH50343]), I get notified that the savedata is corrupt(the same savedata works fine for r3189).
Then I tried with the 32-bit r3315 => same result.
After that, I tried to let the game create a new savegame, but even that new savegame was considered to be corrupted when trying to load. And everytime I let the game delete the newly created corrupted savegame, the emulator goes into endlessly trying to delete something(Savedata MODE_SINGLEDELETE directory not found) after deleting all the files that were created before.

So neither the 32-bit nor the 64-bit r3315 can load my saves from r3189(they were created and used without the crypto-mode) and they can't create new(non-corrupt) savegames as well(the folder and files are created though). Changing to cryto-mode leads to the exact same result.
I don't think the problem is file-system related, since the DLC-data seems to load just fine.
Anyone got an idea what might cause this/how to fix this?

EDIT: r3304 seems to bee the last revision where loading my savegames doesn't result in the claim of "corrupted data"(i guess r3305 would still work if not for the incomplete build). I have he suspicion that the changes introduced in r3306 are the cause for this.

EDIT2: I've confirmend my suspicion. After a rollback to the last version for all the files modified by r3306 loading and saving work again. I think I'll try putting stuff back in litte by little and see what happens.

EDIT4: I now know that whatever breaks loading/saving is in HLE/modules150/sceUtility.java. I'll try modifing that code to make it work.

EDIT5: I think I have the problem: The savefiles I have are secure files[if (saveFileSecureEntriesAddr != 0) results in true]. Furthermore if(saveFileEntriesAddr != 0) results in a false-value. That alone would be no problem but with the new implementation, there is a if(savedataParams.isSecureFile(entry)) check before checking if we have a value for saveFileSecureEntriesAddr. And that check returns a false.
So I took a closer look to the isSecureFile() function. Said function accesses this value: psf.get("SAVEDATA_FILE_LIST"), which is not set for my savefiles.

What exactly is this SAVEDATA_FILE_LIST and why is it suddenly needed, even though things seem to work without it(at least for my saves)?

EDIT6: I forgot to mention, I had to change mem.write64(entryAddr + 8, ((stat.size + 0xF) & ~0xF) - 0x10); back to mem.write64(entryAddr + 8, stat.size); for things to work. What was that change supposed to do?

So now that I found a way to make saving/loading work for me again, could you explain what SAVEDATA_FILE_LIST and ((stat.size + 0xF) & ~0xF) - 0x10) are supposed to be doing gid15? Any idea why that changes made it impossible to load savegames for VC2[ULUS10515] and VC3[NPJH50343]?

[sum2012]

Confirm VC2[ULUS10515] have read save probrem
Attach r3304 and r3307 sceUtility debug log

[gid15]

(07-19-2013 09:30 AM)nightflyer Wrote:  EDIT5: I think I have the problem: The savefiles I have are secure files[if (saveFileSecureEntriesAddr != 0) results in true]. Furthermore if(saveFileEntriesAddr != 0) results in a false-value. That alone would be no problem but with the new implementation, there is a if(savedataParams.isSecureFile(entry)) check before checking if we have a value for saveFileSecureEntriesAddr. And that check returns a false.
So I took a closer look to the isSecureFile() function. Said function accesses this value: psf.get("SAVEDATA_FILE_LIST"), which is not set for my savefiles.

What exactly is this SAVEDATA_FILE_LIST and why is it suddenly needed, even though things seem to work without it(at least for my saves)?

EDIT6: I forgot to mention, I had to change mem.write64(entryAddr + 8, ((stat.size + 0xF) & ~0xF) - 0x10); back to mem.write64(entryAddr + 8, stat.size); for things to work. What was that change supposed to do?

So now that I found a way to make saving/loading work for me again, could you explain what SAVEDATA_FILE_LIST and ((stat.size + 0xF) & ~0xF) - 0x10) are supposed to be doing gid15? Any idea why that changes made it impossible to load savegames for VC2[ULUS10515] and VC3[NPJH50343]?

Thank you for the exact analysis
PARAM.SFO files created before r3306 do not include the entry SAVEDATA_FILE_LIST and are incorrectly interpreted. Could you test again with r3324? Please test the following scenarios:
- loading a savedata created before r3306
- save a new savedata and try to load it (the PARAM.SFO file should now contain the SAVEDATA_FILE_LIST entry)

Thank you!

[sum2012]

Both case still fail in r3324
Log 1 use r3304 save
Log 2 use r3324 save

(07-24-2013 07:26 PM)gid15 Wrote:  PARAM.SFO files created before r3306 do not include the entry SAVEDATA_FILE_LIST and are incorrectly interpreted. Could you test again with r3324? Please test the following scenarios:
- loading a savedata created before r3306
- save a new savedata and try to load it (the PARAM.SFO file should now contain the SAVEDATA_FILE_LIST entry)

Thank you!

[sum2012]

r3325
Log 1 use r3304 save - fail
Log 2 use r3325 save - Sucess
edit:add r3304 save log
#3 only load log

(07-24-2013 10:17 PM)sum2012 Wrote:  Both case still fail in r3324
Log 1 use r3304 save
Log 2 use r3324 save

(07-24-2013 07:26 PM)gid15 Wrote:  PARAM.SFO files created before r3306 do not include the entry SAVEDATA_FILE_LIST and are incorrectly interpreted. Could you test again with r3324? Please test the following scenarios:
- loading a savedata created before r3306
- save a new savedata and try to load it (the PARAM.SFO file should now contain the SAVEDATA_FILE_LIST entry)

Thank you!

[gid15]

(07-25-2013 09:37 PM)sum2012 Wrote:  r3325
Log 1 use r3304 save - fail
Log 2 use r3325 save - Sucess
edit:add r3304 save log
#3 only load log

Jpcsp is using the date of the savedata file to find out if this is a pre-r3306 file or not:
- if date of savedata file < 2013-07-12: file is interpreted as pre-r3306
- if date of savedata file >= 2013-07-12: file is interpreted as r3306 or later

This should work for normal users, but will not work if you specifically do tests now with r3304: e.g. create savedata now with r3304 and try to import them with r3325, they will be interpreted with the new format because their creation date was after Jul 12.

[nightflyer]

(07-26-2013 06:34 AM)gid15 Wrote:  

(07-25-2013 09:37 PM)sum2012 Wrote:  r3325
Log 1 use r3304 save - fail
Log 2 use r3325 save - Sucess
edit:add r3304 save log
#3 only load log

Jpcsp is using the date of the savedata file to find out if this is a pre-r3306 file or not:
- if date of savedata file < 2013-07-12: file is interpreted as pre-r3306
- if date of savedata file >= 2013-07-12: file is interpreted as r3306 or later

This should work for normal users, but will not work if you specifically do tests now with r3304: e.g. create savedata now with r3304 and try to import them with r3325, they will be interpreted with the new format because their creation date was after Jul 12.

Wouldn't it be easier to check if there is a value for saveFileSecureEntriesAddr if there is no SAVEDATA_FILE_LIST? Because if there is, we know it must be a savefile from an old revision right?

Anyway, I tried r3326 and loading worked so far, so good work and thanks.

[sum2012]

Confirm work with save with 1 June 13 with r3304 can be use
in 26-July-13 with r3326
> - if date of savedata file < 2013-07-12: file is interpreted as pre-r3306
> - if date of savedata file >= 2013-07-12: file is interpreted as r3306 or later

[nightflyer]

Since I somehow managed to make my save for VC3[NPJH50343] unreadable/corrupt for r3326, I decided to try my own suggestion and modified isPreR3306 to check if saveFileSecureEntriesAddr != 0 instead of checking the date. Thanks to that, said savegame was now updated to have a valid SAVEDATA_FILE_LIST and can be used with unmodified r3326.

So is there the possibility for there to be a file that is not secure(=has no SAVEDATA_FILE_LIST), has a valid value for saveFileSecureEntriesAddr and is not a valid pre3306-save? Is there a pecific reason why using a date-check is better in this situation?