Yesterday our nexus repository instance went sideways due to a disk full event. After freeing some space and attempting a restart, nexus is no longer accessible. As a result, the database became corrupted (presumably) and cannot be connected to. When I run the nexus-orient-console.jar to connect, here's what I get:
orientdb> CONNECT plocal:/nexus-data/db/component admin admin
Connecting to database [plocal:/nexus-data/db/component] with user 'admin'...
2018-08-09 22:04:03:769 WARNI {db=component} Storage 'component' was not closed properly. Will try to recover from write ahead log...
2018-08-09 22:04:03:775 WARNI {db=component} Record com.orientechnologies.orient.core.storage.impl.local.paginated.wal.OCheckpointEndRecord{lsn=LSN{segment=34, position=52}} will be skipped during data restore$ANSI{green {db=component}} Exception during storage data restore
java.lang.NullPointerException
at com.orientechnologies.orient.core.storage.impl.local.paginated.wal.ODiskWriteAheadLog.cutTill(ODiskWriteAheadLog.java:919)
at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.makeFullCheckpoint(OAbstractPaginatedStorage.java:3718)
at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.recoverIfNeeded(OAbstractPaginatedStorage.java:3920)
at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.open(OAbstractPaginatedStorage.java:294)
at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.open(ODatabaseDocumentTx.java:259)
at com.orientechnologies.orient.console.OConsoleDatabaseApp.connect(OConsoleDatabaseApp.java:234)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.orientechnologies.common.console.OConsoleApplication.execute(OConsoleApplication.java:405)
at com.orientechnologies.common.console.OConsoleApplication.executeCommands(OConsoleApplication.java:260)
at com.orientechnologies.common.console.OConsoleApplication.run(OConsoleApplication.java:131)
at com.orientechnologies.orient.console.OConsoleDatabaseApp.main(OConsoleDatabaseApp.java:145)
at org.sonatype.nexus.orient.console.Main.main(Main.java:58)
Exception `1FDE0371` in storage `plocal:/nexus-data/db/component`: 2.2.31 (build 285537d2767275f460df32c6a3be01bfff6a517c, branch 2.2.x)
java.lang.NullPointerException
at com.orientechnologies.orient.core.storage.impl.local.paginated.wal.ODiskWriteAheadLog.cutTill(ODiskWriteAheadLog.java:919)
at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.makeFullCheckpoint(OAbstractPaginatedStorage.java:3718)
at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.doClose(OAbstractPaginatedStorage.java:4397)
at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.close(OAbstractPaginatedStorage.java:580)
at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.open(OAbstractPaginatedStorage.java:316)
at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.open(ODatabaseDocumentTx.java:259)
at com.orientechnologies.orient.console.OConsoleDatabaseApp.connect(OConsoleDatabaseApp.java:234)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.orientechnologies.common.console.OConsoleApplication.execute(OConsoleApplication.java:405)
at com.orientechnologies.common.console.OConsoleApplication.executeCommands(OConsoleApplication.java:260)
at com.orientechnologies.common.console.OConsoleApplication.run(OConsoleApplication.java:131)
at com.orientechnologies.orient.console.OConsoleDatabaseApp.main(OConsoleDatabaseApp.java:145)
at org.sonatype.nexus.orient.console.Main.main(Main.java:58)
$ANSI{green {db=component}} Error during storage close
java.lang.NullPointerException
at com.orientechnologies.orient.core.storage.impl.local.paginated.wal.ODiskWriteAheadLog.cutTill(ODiskWriteAheadLog.java:919)
at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.makeFullCheckpoint(OAbstractPaginatedStorage.java:3718)
at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.doClose(OAbstractPaginatedStorage.java:4397)
at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.close(OAbstractPaginatedStorage.java:580)
at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.open(OAbstractPaginatedStorage.java:316)
at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.open(ODatabaseDocumentTx.java:259)
at com.orientechnologies.orient.console.OConsoleDatabaseApp.connect(OConsoleDatabaseApp.java:234)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.orientechnologies.common.console.OConsoleApplication.execute(OConsoleApplication.java:405)
at com.orientechnologies.common.console.OConsoleApplication.executeCommands(OConsoleApplication.java:260)
at com.orientechnologies.common.console.OConsoleApplication.run(OConsoleApplication.java:131)
at com.orientechnologies.orient.console.OConsoleDatabaseApp.main(OConsoleDatabaseApp.java:145)
at org.sonatype.nexus.orient.console.Main.main(Main.java:58)
Exception `260A3A5E` in storage `plocal:/nexus-data/db/component`: 2.2.31 (build 285537d2767275f460df32c6a3be01bfff6a517c, branch 2.2.x)
com.orientechnologies.orient.core.exception.OStorageException: Cannot open local storage '/nexus-data/db/component' with mode=rw
DB name="component"
at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.open(OAbstractPaginatedStorage.java:323)
at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.open(ODatabaseDocumentTx.java:259)
at com.orientechnologies.orient.console.OConsoleDatabaseApp.connect(OConsoleDatabaseApp.java:234)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.orientechnologies.common.console.OConsoleApplication.execute(OConsoleApplication.java:405)
at com.orientechnologies.common.console.OConsoleApplication.executeCommands(OConsoleApplication.java:260)
at com.orientechnologies.common.console.OConsoleApplication.run(OConsoleApplication.java:131)
at com.orientechnologies.orient.console.OConsoleDatabaseApp.main(OConsoleDatabaseApp.java:145)
at org.sonatype.nexus.orient.console.Main.main(Main.java:58)
Caused by: java.lang.NullPointerException
at com.orientechnologies.orient.core.storage.impl.local.paginated.wal.ODiskWriteAheadLog.cutTill(ODiskWriteAheadLog.java:919)
at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.makeFullCheckpoint(OAbstractPaginatedStorage.java:3718)
at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.recoverIfNeeded(OAbstractPaginatedStorage.java:3920)
at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.open(OAbstractPaginatedStorage.java:294)
... 11 more
Error: com.orientechnologies.orient.core.exception.OStorageException: Cannot open local storage '/nexus-data/db/component' with mode=rw
DB name="component"
Error: java.lang.NullPointerException
Because I am not able to even connect to the database, I haven't been able to attempt the recovery steps suggested here > OrientDB corruption state in Nexus Repository version 3.2.0-01. Is there any way to repair and recover the databases or are we just S.O.L.? Is there any way to start nexus in a safe-mode that ignores database issues?
This thread must fixed our issue. https://groups.google.com/a/glists.sonatype.com/g/nexus-users/c/64W_y9fDGzQ/m/o0FKRgiDAwAJ?pli=1
To summarize, this steps fixed my error (same as you) :
rm /mnt/sonatype-work/nexus3/db/config/*.wal
Then try to start the console and repair the database
At the orientdb prompt run the following:
connect plocal:/mnt/sonatype-work/nexus3/db/confg admin admin
repair database --fix-links
rebuild index *
disconnect
exit
For Nexus 3.2.1+ it is possible to use the included OrientDB client.