Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Artifactory warns about "Path checksum calculation job" after upgrade to 5.8.3

I just migrated our OSS instance of Artifactory from 5.4.5 to 5.8.3 (standalone, using Derby)

I followed the recommendations at https://www.jfrog.com/confluence/display/RTF/Upgrading+Artifactory and basically did:

  • Actual upgrade to 5.8.3 (stop, replace some files, start)
  • SHA-256 job to calculate checksums on existing artifacts (stop, add property to confiuguration, start)

Both worked fine and the server is back up and running smoothly.

However, I now have a warning in logs/artifactory.log on instance start up

2018-01-24 16:12:07,633 [art-exec-4] [WARN ] (o.a.s.j.m.p.RepoPathChecksumMigrationJobDelegate:110) - Path Checksum calculation job (for existing artifacts) has been disabled and will not run, there are still 5348 artifacts without path checksum values in the database. Future version of Artifactory may enforce this conversion as a prerequisite for upgrades.

I cannot find any more substantial explanation for this warning.

I am thinking that this is related to the artifacts layout on the file system (cf. https://www.jfrog.com/confluence/display/RTF/Checksum-Based+Storage#Checksum-BasedStorage-Overview). The artifacts are still laid according to their SHA-1checksums rather than their SHA-256 checksums.

Is my assumption correct? How do I "fix" this warning...

EDIT: Some more tests requested by @Ariel:

  • Restarting the server doesn't help, the warning is still there
  • Reenabling the migration job and restarting the server doesn't help either

echo "artifactory.sha2.migration.job.enabled=true" >> etc/artifactory.system.properties

Looking at the logs related to this migration, it seems that the migration job thinks that everything has been migrated, and the startup check finds 2,000 artifacts that should be migrated.

$ARTIFACTORY_HOME/logs/sha256_migration.log

2018-01-24 14:39:53,982 [art-exec-3] [INFO ] (o.a.s.j.m.s.Sha256MigrationJob:284) - artifactory.Sha256MigrationJob#83253c93-33ec-4c52-bc61-d2d33942dc28: all nodes reached minimal version '5.5.0-m001', continuing execution
2018-01-24 16:12:07,576 [art-exec-3] [INFO ] (o.a.s.j.m.s.Sha256MigrationJob:284) - artifactory.Sha256MigrationJob#6f5c6739-c365-4be2-80a4-d32063a75f8f: all nodes reached minimal version '5.5.0-m001', continuing execution
2018-01-24 16:12:07,651 [art-exec-3] [INFO ] (o.a.s.j.m.s.Sha256MigrationJob:186) - 3319 artifacts and 3292 binary entries are missing SHA256 values - starting calculation job.
2018-01-24 16:12:07,756 [art-exec-3] [INFO ] (o.a.s.j.m.s.Sha256MigrationJob:121) - SHA256 migration state: 0/3319 artifacts were handled.
[...]
2018-01-24 16:13:58,226 [art-exec-3] [INFO ] (o.a.s.j.m.s.Sha256MigrationJob:121) - SHA256 migration state: 3318/3319 artifacts were handled.
2018-01-24 16:13:58,227 [art-exec-3] [INFO ] (o.a.s.j.m.s.Sha256MigrationJob:270) - SHA256 migration job now filling in for missing SHA256 values for binary entries. Found 1 such entries
2018-01-24 16:14:01,065 [art-exec-3] [INFO ] (o.a.s.j.m.s.Sha256MigrationJob:338) - SHA256 migration job has finished successfully. 3319 artifacts and 1 binary entry calculations were submitted (including retries)
2018-01-24 16:29:06,072 [art-exec-3] [INFO ] (o.a.s.j.m.s.Sha256MigrationJob:284) - artifactory.Sha256MigrationJob#7cdeab66-229d-43a5-a788-301f72c10cc5: all nodes reached minimal version '5.5.0-m001', continuing execution
2018-01-29 06:04:57,405 [art-exec-3] [INFO ] (o.a.s.j.m.s.Sha256MigrationJob:284) - artifactory.Sha256MigrationJob#2c539185-b7f4-412f-b988-0688e8505649: all nodes reached minimal version '5.5.0-m001', continuing execution
2018-01-29 10:27:33,655 [art-exec-3] [INFO ] (o.a.s.j.m.s.Sha256MigrationJob:284) - artifactory.Sha256MigrationJob#19afc5d2-c12d-4821-8c1a-808655e8746c: all nodes reached minimal version '5.5.0-m001', continuing execution
2018-01-29 10:31:39,250 [art-exec-3] [INFO ] (o.a.s.j.m.s.Sha256MigrationJob:284) - artifactory.Sha256MigrationJob#c01bd20d-249a-4c6e-80ff-e26301db7e84: all nodes reached minimal version '5.5.0-m001', continuing execution
2018-01-29 10:34:57,321 [art-exec-3] [INFO ] (o.a.s.j.m.s.Sha256MigrationJob:284) - artifactory.Sha256MigrationJob#32fff21f-04d5-45a4-84d2-58083aaf6593: all nodes reached minimal version '5.5.0-m001', continuing execution

$ARTIFACTORY_HOME/logs/path_checksum_migration.log

2018-01-24 14:39:53,982 [art-exec-4] [INFO ] (o.a.s.j.m.p.RepoPathChecksumMigrationJob:284) - artifactory.RepoPathChecksumMigrationJob#2f7f690c-dc78-4074-b35f-e5085d41a2f7: all nodes reached minimal version '5.5.0-m001', continuing execution
2018-01-24 14:39:54,012 [art-exec-4] [WARN ] (o.a.s.j.m.p.RepoPathChecksumMigrationJob:111) - Path Checksum calculation job (for existing artifacts) has been disabled and will not run, there are still 5348 artifacts without path checksum values in the database. Future version of Artifactory may enforce this conversion as a prerequisite for upgrades.
2018-01-24 16:12:07,576 [art-exec-4] [INFO ] (o.a.s.j.m.p.RepoPathChecksumMigrationJob:284) - artifactory.RepoPathChecksumMigrationJob#4c3a71ed-f389-4bff-a7b9-62d20806b270: all nodes reached minimal version '5.5.0-m001', continuing execution
2018-01-24 16:12:07,634 [art-exec-4] [WARN ] (o.a.s.j.m.p.RepoPathChecksumMigrationJob:111) - Path Checksum calculation job (for existing artifacts) has been disabled and will not run, there are still 5348 artifacts without path checksum values in the database. Future version of Artifactory may enforce this conversion as a prerequisite for upgrades.
2018-01-24 16:29:06,072 [art-exec-4] [INFO ] (o.a.s.j.m.p.RepoPathChecksumMigrationJob:284) - artifactory.RepoPathChecksumMigrationJob#8e2c1a01-c3d9-4848-b48a-70813ffd26d1: all nodes reached minimal version '5.5.0-m001', continuing execution
2018-01-24 16:29:06,127 [art-exec-4] [WARN ] (o.a.s.j.m.p.RepoPathChecksumMigrationJob:111) - Path Checksum calculation job (for existing artifacts) has been disabled and will not run, there are still 2029 artifacts without path checksum values in the database. Future version of Artifactory may enforce this conversion as a prerequisite for upgrades.
2018-01-29 06:04:57,405 [art-exec-4] [INFO ] (o.a.s.j.m.p.RepoPathChecksumMigrationJob:284) - artifactory.RepoPathChecksumMigrationJob#b7ac1c4a-5dec-4065-a901-bb5a3d2a4b59: all nodes reached minimal version '5.5.0-m001', continuing execution
2018-01-29 06:04:57,505 [art-exec-4] [WARN ] (o.a.s.j.m.p.RepoPathChecksumMigrationJob:111) - Path Checksum calculation job (for existing artifacts) has been disabled and will not run, there are still 2029 artifacts without path checksum values in the database. Future version of Artifactory may enforce this conversion as a prerequisite for upgrades.
2018-01-29 10:27:33,655 [art-exec-4] [INFO ] (o.a.s.j.m.p.RepoPathChecksumMigrationJob:284) - artifactory.RepoPathChecksumMigrationJob#74cf399a-1c3b-4b11-a687-cc11b19d2887: all nodes reached minimal version '5.5.0-m001', continuing execution
2018-01-29 10:27:33,704 [art-exec-4] [WARN ] (o.a.s.j.m.p.RepoPathChecksumMigrationJob:111) - Path Checksum calculation job (for existing artifacts) has been disabled and will not run, there are still 2029 artifacts without path checksum values in the database. Future version of Artifactory may enforce this conversion as a prerequisite for upgrades.
2018-01-29 10:31:39,250 [art-exec-4] [INFO ] (o.a.s.j.m.p.RepoPathChecksumMigrationJob:284) - artifactory.RepoPathChecksumMigrationJob#ba1c5406-2f49-48f2-a9f2-a9e48c8d7807: all nodes reached minimal version '5.5.0-m001', continuing execution
2018-01-29 10:31:39,308 [art-exec-4] [WARN ] (o.a.s.j.m.p.RepoPathChecksumMigrationJob:111) - Path Checksum calculation job (for existing artifacts) has been disabled and will not run, there are still 2029 artifacts without path checksum values in the database. Future version of Artifactory may enforce this conversion as a prerequisite for upgrades.
2018-01-29 10:34:57,321 [art-exec-4] [INFO ] (o.a.s.j.m.p.RepoPathChecksumMigrationJob:284) - artifactory.RepoPathChecksumMigrationJob#b62872c1-4c00-4503-8628-bc2dd38d8c17: all nodes reached minimal version '5.5.0-m001', continuing execution
2018-01-29 10:34:57,372 [art-exec-4] [WARN ] (o.a.s.j.m.p.RepoPathChecksumMigrationJob:111) - Path Checksum calculation job (for existing artifacts) has been disabled and will not run, there are still 2029 artifacts without path checksum values in the database. Future version of Artifactory may enforce this conversion as a prerequisite for upgrades.
like image 830
Arnaud Jeansen Avatar asked Jan 26 '18 08:01

Arnaud Jeansen


People also ask

How to check the status of the Artifactory migration?

Check the Migration Log and review system.yaml to validate the migration was successful (only for upgrading from v6.x). Start the service using the start scripts or OS service management. Check the Artifactory Log for the status of the service. The default Artifactory home directory is defined according to the installation type.

How do I check for errors in Artifactory?

In case of an error you will see an error message in the page header. Access Artifactory through your load balancer and log in as Admin. In the Admin module go to Monitoring | Service Status.

Why does Artifactory keep coming up with older version?

If Artifactory was installed as a service in previous version, install this version also as a service. When an earlier version is installed as a service, it is important to update the new one also as a service. Otherwise a restart of the server may lead to older version of Artifactory coming up.

How do I upgrade to Artifactory 7?

When upgrading to Artifactory 7.11.x, you need Mission Control 4.6.x to be able to use the new metrics and trends. Stop the current server. Extract the contents of the compressed archive and go to the extracted folder. Replace the existing $JFROG_HOME/artifactory/app with the new app folder.


2 Answers

This is poorly documented — I had to download the Artifactory OSS source code and grep for the answer — but apparently Path Checksum migration is a completely separate job from SHA-256 migration. You want artifactory.pathChecksum.migration.job.enabled=true in your artifactory.system.properties, I think.

like image 134
Donald King Avatar answered Sep 28 '22 03:09

Donald King


The operation of Sha-256 is not being done automatically. You need to manually set it up so it will run. This was done in order not to cause overload on users environments. If you wish to activate it follow this link: https://www.jfrog.com/confluence/display/RTF/Checksum-Based+Storage#Checksum-BasedStorage-MigratingtheDatabasetoIncludeSHA-256

like image 29
Ariel Avatar answered Sep 28 '22 01:09

Ariel