Savedata Encryption/Decryption

[popsdeco]

Oh, r3412 works partly?
Then I can assume that there will be no drastic change like r3408. time to do stuffs maybe.

[popsdeco]

https://github.com/cielavenir/psp-saveda...its/master
This stuff doesn't recalculate the same hash from the same PARAM.SFO yet...

[Edit] What I mean is, now that it is known that SAVEDATA_PARAMS depends only on PARAM.SFO,
1. clear SAVEDATA_PARAMS
2. recalculate SAVEDATA_PARAMS from current PARAM.SFO
should generate the same PARAM.SFO.

[onelight]

SAVEDATA encryption is now enabled by default, so the option in Crypto is "Disbable SAVEDATA in crypto mode", but in 32bit jpcsp, it still is "Handle SAVEDATA in crypto mode" , you mine fotget change it. and so other language.

Test JPCSP 3433 MHP3: can read encrypted SAVEDATA files but couldnot savedata in crypto mode. you didnot fix it.

Code:

39621 [user_main] INFO  hle.sceAtrac3plus - Decodable AT3 data detected.
39622 [user_main] INFO  hle.sceAtrac3plus - Playing AT3 file 'MemoryVirtualFile 0x08DA9440-0x08DC0120 (length=0x16CE0)'
39692 [user_main] INFO  compiler - Splitting _S1_2_88941B4 (132/50)
39734 [user_main] INFO  compiler - Splitting _S1_2_88940EC (158/50)
39844 [calcSumThread] INFO  compiler - Splitting _S1_2_888F640 (60/50)
39972 [user_main] INFO  hle.sceUtility - sceUtilitySavedataInitStart 0x09BAEA10-0x09BAF010: Address 0x09BAEA10, mode=1(AUTOSAVE), gameName=ULJM05800, saveName=<>, fileName=MHP3RD.BIN, secureVersion=0
40019 [user_main] ERROR hle.sceUtility - java.lang.ArrayIndexOutOfBoundsException
40036 [user_main] INFO  compiler - Splitting _S1_2_88942E8 (135/50)
40082 [user_main] ERROR hle.sceUtility - java.lang.ArrayIndexOutOfBoundsException
40128 [user_main] ERROR hle.sceUtility - java.lang.ArrayIndexOutOfBoundsException
40174 [user_main] ERROR hle.sceUtility - java.lang.ArrayIndexOutOfBoundsException
40225 [user_main] ERROR hle.sceUtility - java.lang.ArrayIndexOutOfBoundsException
40278 [user_main] ERROR hle.sceUtility - java.lang.ArrayIndexOutOfBoundsException
40325 [user_main] ERROR hle.sceUtility - java.lang.ArrayIndexOutOfBoundsException
40402 [user_main] ERROR hle.sceUtility - java.lang.ArrayIndexOutOfBoundsException
40459 [user_main] ERROR hle.sceUtility - java.lang.ArrayIndexOutOfBoundsException
40507 [user_main] ERROR hle.sceUtility - java.lang.ArrayIndexOutOfBoundsException
40623 [user_main] INFO  hle.sceUtility - sceUtilityMsgDialogInitStart 0x09BAF080-0x09BAF344: result 0x00000000
mode PSP_UTILITY_MSGDIALOG_MODE_ERROR
errorValue 0x80110385
message ''
options 0x00000000
buttonPressed 0x00000000
42141 [user_main] INFO  compiler - Splitting _S1_2_9CEEC20 (327/50)
42147 [user_main] INFO  compiler - Splitting _S1_2_9CEE94C (378/50)
42150 [user_main] INFO  compiler - Splitting _S1_2_9CEE6F4 (150/50)
42160 [user_main] INFO  compiler - Splitting _S1_2_9CFD848 (1563/50)
42195 [user_main] INFO  compiler - Splitting _S1_2_9CF9650 (147/50)
42580 [user_main] INFO  compiler - Splitting _S1_2_9C06CB0 (100/50)
42584 [user_main] INFO  compiler - Splitting _S1_2_8912788 (209/50)
42587 [user_main] INFO  compiler - Splitting _S1_2_9C0D66C (75/50)
42590 [user_main] INFO  compiler - Splitting _S1_2_8912B2C (170/50)
42626 [user_main] INFO  compiler - Splitting _S1_2_9C0B2C0 (431/50)
45351 [user_main] INFO  compiler - Splitting _S1_2_9CF9564 (140/50)
47287 [user_main] INFO  compiler - Splitting _S1_2_9C06E40 (111/50)
50405 [user_main] INFO  compiler - Splitting _S1_2_9CF9138 (127/50)
50873 [user_main] INFO  compiler - Splitting _S1_2_9C2D0F0 (301/50)

mhp3 hle.sceUtility trace log.zip

  mhp3 hle.sceUtility trace log.zip (Size: 7.9 KB / Downloads: 90)

[Hykem]

As requested, I've added a better handling of SAVEDATA files.
As of r3435 JPCSP looks for a specific bit on the PARAM.SFO file (a similar check is performed on a real PSP) and auto-detects the SAVEDATA format, so you no longer need to enable/disable any options.

The "Disable SAVEDATA encryption" option is still there, but it's now oriented for developers to test data.

Without changing any option, JPCSP does the following:
Plain data (unencrypted) detected -> Load the data directly -> Data is changed by the application -> Write encrypted data
Encrypted data detected -> Decrypt and load the data -> Data is changed by the application -> Write re-encrypted data

But, if you check the "Disable SAVEDATA encryption", JPCSP does:
Plain data (unencrypted) detected -> Load the data directly -> Data is changed by the application -> Write plain data (unencrypted)
Encrypted data detected -> Decrypt and load the data -> Data is changed by the application -> Write plain data (unencrypted)

This way, you can force JPCSP to output decrypted data, which means you can use JPCSP to output your SAVEDATA in encrypted or decrypted format.
Combining this with the "Extract SAVEDATA key" options gives you all the necessary tools to modify, decrypt and encrypt your SAVEDATA, either with JPCSP, PPSSPP, or any external tools.

I've also improved the algorithm and the hashing routines, fixed Java issues and a few other bugs. I would really appreciate any reports on these new changes, specifically if there are any improvements with PSP compatibility. Thanks!

[akaya]

I hope this thread is a right place to report savedata error.
Recently Jpcsp gives an error and freezes when I try to load savedata on the particular game, SoraYume.
Saving works, but loading it gives an error.
Saving and loading all work fine with older versions, r3200 and r3210.
I deleted all previous saves and made new savefile with recent version, but it did not solve the error.
The red font error message on log level:INFO on r3435 is this:

113422 [user_main] ERROR runtime - Catched Throwable in RuntimeThread:
java.lang.ArrayIndexOutOfBoundsException: 41943040 at jpcsp.memory.MemoryReader$MemoryReaderIntArray8.readNext(MemoryReader.java:210) at jpcsp.util.MemoryInputStream.read(MemoryInputStream.java:44) at javax.imageio.stream.FileCacheImageInputStream.readUntil(Unknown Source) at javax.imageio.stream.FileCacheImageInputStream.read(Unknown Source) at javax.imageio.stream.ImageInputStreamImpl.readInt(Unknown Source) at com.sun.imageio.plugins.png.PNGImageReader.readMetadata(Unknown Source) at com.sun.imageio.plugins.png.PNGImageReader.readImage(Unknown Source) at com.sun.imageio.plugins.png.PNGImageReader.read(Unknown Source) at javax.imageio.ImageIO.read(Unknown Source) at javax.imageio.ImageIO.read(Unknown Source) at jpcsp.HLE.modules150.sceUtility$GuUtilityDialog.readIcon(sceUtility.java:2729) at jpcsp.HLE.modules150.sceUtility$GuUtilityDialog.readIcon(sceUtility.java:2779) at jpcsp.HLE.modules150.sceUtility$GuSavedataDialogLoad.updateDialog(sceUtility.java:2886) at jpcsp.HLE.modules150.sceUtility$GuUtilityDialog.update(sceUtility.java:2284) at jpcsp.HLE.modules150.sceUtility$UtilityDialogState.executeUpdate(sceUtility.java:506) at jpcsp.HLE.modules150.sceUtility.sceUtilitySavedataUpdate(sceUtility.java:3518) at _S1_2_886ECF8.s(_S1_2_886ECF8.java:4) at _S1_2_881258C.s8812b70(_S1_2_881258C.java:1524) at _S1_2_881258C.s(_S1_2_881258C.java:1508) at _S1_2_881391C.s(_S1_2_881391C.java:20) at _S1_2_8818908.s(_S1_2_8818908.java:88) at _S1_2_88375A4.s8837a04(_S1_2_88375A4.java) at _S1_2_88375A4.s(_S1_2_88375A4.java:35912) at _S1_2_88375A4.exec(_S1_2_88375A4.java) at jpcsp.Allegrex.compiler.RuntimeContext.jumpCall(RuntimeContext.java:157) at jpcsp.Allegrex.compiler.RuntimeContext.call(RuntimeContext.java:210) at _S1_2_8808398.s(_S1_2_8808398.java:56) at _S1_2_8804BD8.s8804e84(_S1_2_8804BD8.java:688) at _S1_2_8804BD8.s(_S1_2_8804BD8.java:684) at _S1_2_8804238.s8804340(_S1_2_8804238.java:268) at _S1_2_8804238.s(_S1_2_8804238.java:264) at _S1_2_8804238.exec(_S1_2_8804238.java) at jpcsp.Allegrex.compiler.RuntimeContext.execWithReturnAddress(RuntimeContext.java:668) at jpcsp.Allegrex.compiler.RuntimeContext.runThread(RuntimeContext.java:709) at jpcsp.Allegrex.compiler.RuntimeThread.run(RuntimeThread.java:50)

Thank you.

[onelight]

As requested, I've added a better handling of SAVEDATA files.
As of r3435 JPCSP looks for a specific bit on the PARAM.SFO file (a similar check is performed on a real PSP) and auto-detects the SAVEDATA format, so you no longer need to enable/disable any options.

The "Disable SAVEDATA encryption" option is still there, but it's now oriented for developers to test data.

Without changing any option, JPCSP does the following:
Plain data (unencrypted) detected -> Load the data directly -> Data is changed by the application -> Write encrypted data
Encrypted data detected -> Decrypt and load the data -> Data is changed by the application -> Write re-encrypted data

But, if you check the "Disable SAVEDATA encryption", JPCSP does:
Plain data (unencrypted) detected -> Load the data directly -> Data is changed by the application -> Write plain data (unencrypted)
Encrypted data detected -> Decrypt and load the data -> Data is changed by the application -> Write plain data (unencrypted)

This way, you can force JPCSP to output decrypted data, which means you can use JPCSP to output your SAVEDATA in encrypted or decrypted format.
Combining this with the "Extract SAVEDATA key" options gives you all the necessary tools to modify, decrypt and encrypt your SAVEDATA, either with JPCSP, PPSSPP, or any external tools.

I've also improved the algorithm and the hashing routines, fixed Java issues and a few other bugs. I would really appreciate any reports on these new changes, specifically if there are any improvements with PSP compatibility. Thanks!

Think you, now mhp3 can save and load both unencrypted and encrypted.
But,PSP still need MagicSave to read JPCSP savedata (encrypted), It may be the PARAM.SFO file generate by jpcsp is not the same PARAM.SFO file generate by psp
   

SAVEDATA for you,
NPJH50465MAIN00 is unencrypted and NPJH50465MAIN01 is encrypted
JPCSP
  JPCSP SAVEDATA.zip (Size: 566.91 KB / Downloads: 80)
PSP
  PSP SAVEDATA.zip (Size: 566.96 KB / Downloads: 100)
PPSSPP
  PPSSPP SAVEDATA.zip (Size: 566.87 KB / Downloads: 102)

[Hykem]

As requested, I've added a better handling of SAVEDATA files.
As of r3435 JPCSP looks for a specific bit on the PARAM.SFO file (a similar check is performed on a real PSP) and auto-detects the SAVEDATA format, so you no longer need to enable/disable any options.

The "Disable SAVEDATA encryption" option is still there, but it's now oriented for developers to test data.

Without changing any option, JPCSP does the following:
Plain data (unencrypted) detected -> Load the data directly -> Data is changed by the application -> Write encrypted data
Encrypted data detected -> Decrypt and load the data -> Data is changed by the application -> Write re-encrypted data

But, if you check the "Disable SAVEDATA encryption", JPCSP does:
Plain data (unencrypted) detected -> Load the data directly -> Data is changed by the application -> Write plain data (unencrypted)
Encrypted data detected -> Decrypt and load the data -> Data is changed by the application -> Write plain data (unencrypted)

This way, you can force JPCSP to output decrypted data, which means you can use JPCSP to output your SAVEDATA in encrypted or decrypted format.
Combining this with the "Extract SAVEDATA key" options gives you all the necessary tools to modify, decrypt and encrypt your SAVEDATA, either with JPCSP, PPSSPP, or any external tools.

I've also improved the algorithm and the hashing routines, fixed Java issues and a few other bugs. I would really appreciate any reports on these new changes, specifically if there are any improvements with PSP compatibility. Thanks!

Think you, now mhp3 can save and load both unencrypted and encrypted.
But,PSP still need MagicSave to read JPCSP savedata (encrypted), It may be the PARAM.SFO file generate by jpcsp is not the same PARAM.SFO file generate by psp


SAVEDATA for you,
NPJH50465MAIN00 is unencrypted and NPJH50465MAIN01 is encrypted
JPCSP
PSP
PPSSPP

Ah, yes. From the SAVEDATA you posted it seems to be an issue with the PARAM.SFO. I think the hashes are being generated correctly, but I'm not sending the right data to hash. Thanks!

I hope this thread is a right place to report savedata error.
Recently Jpcsp gives an error and freezes when I try to load savedata on the particular game, SoraYume.
Saving works, but loading it gives an error.
Saving and loading all work fine with older versions, r3200 and r3210.
I deleted all previous saves and made new savefile with recent version, but it did not solve the error.
The red font error message on log level:INFO on r3435 is this:

113422 [user_main] ERROR runtime - Catched Throwable in RuntimeThread:
java.lang.ArrayIndexOutOfBoundsException: 41943040 at jpcsp.memory.MemoryReader$MemoryReaderIntArray8.readNext(MemoryReader.java:210) at jpcsp.util.MemoryInputStream.read(MemoryInputStream.java:44) at javax.imageio.stream.FileCacheImageInputStream.readUntil(Unknown Source) at javax.imageio.stream.FileCacheImageInputStream.read(Unknown Source) at javax.imageio.stream.ImageInputStreamImpl.readInt(Unknown Source) at com.sun.imageio.plugins.png.PNGImageReader.readMetadata(Unknown Source) at com.sun.imageio.plugins.png.PNGImageReader.readImage(Unknown Source) at com.sun.imageio.plugins.png.PNGImageReader.read(Unknown Source) at javax.imageio.ImageIO.read(Unknown Source) at javax.imageio.ImageIO.read(Unknown Source) at jpcsp.HLE.modules150.sceUtility$GuUtilityDialog.readIcon(sceUtility.java:2729) at jpcsp.HLE.modules150.sceUtility$GuUtilityDialog.readIcon(sceUtility.java:2779) at jpcsp.HLE.modules150.sceUtility$GuSavedataDialogLoad.updateDialog(sceUtility.java:2886) at jpcsp.HLE.modules150.sceUtility$GuUtilityDialog.update(sceUtility.java:2284) at jpcsp.HLE.modules150.sceUtility$UtilityDialogState.executeUpdate(sceUtility.java:506) at jpcsp.HLE.modules150.sceUtility.sceUtilitySavedataUpdate(sceUtility.java:3518) at _S1_2_886ECF8.s(_S1_2_886ECF8.java:4) at _S1_2_881258C.s8812b70(_S1_2_881258C.java:1524) at _S1_2_881258C.s(_S1_2_881258C.java:1508) at _S1_2_881391C.s(_S1_2_881391C.java:20) at _S1_2_8818908.s(_S1_2_8818908.java:88) at _S1_2_88375A4.s8837a04(_S1_2_88375A4.java) at _S1_2_88375A4.s(_S1_2_88375A4.java:35912) at _S1_2_88375A4.exec(_S1_2_88375A4.java) at jpcsp.Allegrex.compiler.RuntimeContext.jumpCall(RuntimeContext.java:157) at jpcsp.Allegrex.compiler.RuntimeContext.call(RuntimeContext.java:210) at _S1_2_8808398.s(_S1_2_8808398.java:56) at _S1_2_8804BD8.s8804e84(_S1_2_8804BD8.java:688) at _S1_2_8804BD8.s(_S1_2_8804BD8.java:684) at _S1_2_8804238.s8804340(_S1_2_8804238.java:268) at _S1_2_8804238.s(_S1_2_8804238.java:264) at _S1_2_8804238.exec(_S1_2_8804238.java) at jpcsp.Allegrex.compiler.RuntimeContext.execWithReturnAddress(RuntimeContext.java:668) at jpcsp.Allegrex.compiler.RuntimeContext.runThread(RuntimeContext.java:709) at jpcsp.Allegrex.compiler.RuntimeThread.run(RuntimeThread.java:50)

Thank you.

Hmm, it looks like this game is missing the SAVEDATA ICON image. Could you please tell me which files are inside the game's SAVEDATA folder? Thanks!

[akaya]

As requested, I've added a better handling of SAVEDATA files.
As of r3435 JPCSP looks for a specific bit on the PARAM.SFO file (a similar check is performed on a real PSP) and auto-detects the SAVEDATA format, so you no longer need to enable/disable any options.

The "Disable SAVEDATA encryption" option is still there, but it's now oriented for developers to test data.

Without changing any option, JPCSP does the following:
Plain data (unencrypted) detected -> Load the data directly -> Data is changed by the application -> Write encrypted data
Encrypted data detected -> Decrypt and load the data -> Data is changed by the application -> Write re-encrypted data

But, if you check the "Disable SAVEDATA encryption", JPCSP does:
Plain data (unencrypted) detected -> Load the data directly -> Data is changed by the application -> Write plain data (unencrypted)
Encrypted data detected -> Decrypt and load the data -> Data is changed by the application -> Write plain data (unencrypted)

This way, you can force JPCSP to output decrypted data, which means you can use JPCSP to output your SAVEDATA in encrypted or decrypted format.
Combining this with the "Extract SAVEDATA key" options gives you all the necessary tools to modify, decrypt and encrypt your SAVEDATA, either with JPCSP, PPSSPP, or any external tools.

I've also improved the algorithm and the hashing routines, fixed Java issues and a few other bugs. I would really appreciate any reports on these new changes, specifically if there are any improvements with PSP compatibility. Thanks!

Think you, now mhp3 can save and load both unencrypted and encrypted.
But,PSP still need MagicSave to read JPCSP savedata (encrypted), It may be the PARAM.SFO file generate by jpcsp is not the same PARAM.SFO file generate by psp


SAVEDATA for you,
NPJH50465MAIN00 is unencrypted and NPJH50465MAIN01 is encrypted
JPCSP
PSP
PPSSPP

Ah, yes. From the SAVEDATA you posted it seems to be an issue with the PARAM.SFO. I think the hashes are being generated correctly, but I'm not sending the right data to hash. Thanks!

I hope this thread is a right place to report savedata error.
Recently Jpcsp gives an error and freezes when I try to load savedata on the particular game, SoraYume.
Saving works, but loading it gives an error.
Saving and loading all work fine with older versions, r3200 and r3210.
I deleted all previous saves and made new savefile with recent version, but it did not solve the error.
The red font error message on log level:INFO on r3435 is this:

113422 [user_main] ERROR runtime - Catched Throwable in RuntimeThread:
java.lang.ArrayIndexOutOfBoundsException: 41943040 at jpcsp.memory.MemoryReader$MemoryReaderIntArray8.readNext(MemoryReader.java:210) at jpcsp.util.MemoryInputStream.read(MemoryInputStream.java:44) at javax.imageio.stream.FileCacheImageInputStream.readUntil(Unknown Source) at javax.imageio.stream.FileCacheImageInputStream.read(Unknown Source) at javax.imageio.stream.ImageInputStreamImpl.readInt(Unknown Source) at com.sun.imageio.plugins.png.PNGImageReader.readMetadata(Unknown Source) at com.sun.imageio.plugins.png.PNGImageReader.readImage(Unknown Source) at com.sun.imageio.plugins.png.PNGImageReader.read(Unknown Source) at javax.imageio.ImageIO.read(Unknown Source) at javax.imageio.ImageIO.read(Unknown Source) at jpcsp.HLE.modules150.sceUtility$GuUtilityDialog.readIcon(sceUtility.java:2729) at jpcsp.HLE.modules150.sceUtility$GuUtilityDialog.readIcon(sceUtility.java:2779) at jpcsp.HLE.modules150.sceUtility$GuSavedataDialogLoad.updateDialog(sceUtility.java:2886) at jpcsp.HLE.modules150.sceUtility$GuUtilityDialog.update(sceUtility.java:2284) at jpcsp.HLE.modules150.sceUtility$UtilityDialogState.executeUpdate(sceUtility.java:506) at jpcsp.HLE.modules150.sceUtility.sceUtilitySavedataUpdate(sceUtility.java:3518) at _S1_2_886ECF8.s(_S1_2_886ECF8.java:4) at _S1_2_881258C.s8812b70(_S1_2_881258C.java:1524) at _S1_2_881258C.s(_S1_2_881258C.java:1508) at _S1_2_881391C.s(_S1_2_881391C.java:20) at _S1_2_8818908.s(_S1_2_8818908.java:88) at _S1_2_88375A4.s8837a04(_S1_2_88375A4.java) at _S1_2_88375A4.s(_S1_2_88375A4.java:35912) at _S1_2_88375A4.exec(_S1_2_88375A4.java) at jpcsp.Allegrex.compiler.RuntimeContext.jumpCall(RuntimeContext.java:157) at jpcsp.Allegrex.compiler.RuntimeContext.call(RuntimeContext.java:210) at _S1_2_8808398.s(_S1_2_8808398.java:56) at _S1_2_8804BD8.s8804e84(_S1_2_8804BD8.java:688) at _S1_2_8804BD8.s(_S1_2_8804BD8.java:684) at _S1_2_8804238.s8804340(_S1_2_8804238.java:268) at _S1_2_8804238.s(_S1_2_8804238.java:264) at _S1_2_8804238.exec(_S1_2_8804238.java) at jpcsp.Allegrex.compiler.RuntimeContext.execWithReturnAddress(RuntimeContext.java:668) at jpcsp.Allegrex.compiler.RuntimeContext.runThread(RuntimeContext.java:709) at jpcsp.Allegrex.compiler.RuntimeThread.run(RuntimeThread.java:50)

Thank you.

Hmm, it looks like this game is missing the SAVEDATA ICON image. Could you please tell me which files are inside the game's SAVEDATA folder? Thanks!

The attached files are SoraYume savedata from r3435 and r3210.
_0000 is the systemdata and _0001 is normal game save.
Loading old(r3210) savedata in r3435 gives the same error, but loading new(r3435) save in r3210 crypto mode works without a problem.
Thank you.

[Hykem]

I've fixed a few more bugs in sceUtility that could be related to this issue. Could you please check again? Thanks!

[akaya]

I've fixed a few more bugs in sceUtility that could be related to this issue. Could you please check again? Thanks!

In Automated Jpcsp builds page, r3436 has no build.
Could you check if the file is correctly uploaded? Thanks!