Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Schema mismatch right after install #1195

Open
maringuu opened this issue Feb 16, 2024 · 4 comments
Open

Schema mismatch right after install #1195

maringuu opened this issue Feb 16, 2024 · 4 comments
Labels
bug

Comments

@maringuu
Copy link
Collaborator

FACT version

4.1.1

Environment

Ubuntu 20.04.06 LTS

Steps to reproduce

  1. Install FACT
  2. Run the backend
  3. FACt will complain about a schema mismatch, despite the fact that it is freshly installed

Expeced Behavior

It works.

Installation logs

No response

Backend logs

[2024-02-16 15:43:36][migration][INFO]: Context impl PostgresqlImpl.
[2024-02-16 15:43:36][migration][INFO]: Will assume transactional DDL.
[2024-02-16 15:43:36][fact_base][ERROR]: The database schema in "storage/schema.py" does not match the schema of the configured database. Please run `alembic upgrade head` from `src` to update the schema.
Traceback (most recent call last):
  File "./start_fact_backend", line 125, in <module>
    FactBackend().main()
  File "./start_fact_backend", line 48, in __init__
    super().__init__()
  File "/home/maringuu/opt/FACT_core-v4.1.1/src/fact_base.py", line 40, in __init__
    self.do_self_test()
  File "/home/maringuu/opt/FACT_core-v4.1.1/src/fact_base.py", line 88, in do_self_test
    raise DbInterfaceError('Schema mismatch')
storage.db_interface_base.DbInterfaceError: Schema mismatch

Frontend logs

No response

Other information

No response
@maringuu maringuu added the bug label Feb 16, 2024
@SpaceMoehre
Copy link

Same here, Installation runs smoothly and no obvious errors but db fails:

user@fact:~$ uname -a
Linux fact 5.4.0-177-generic #197-Ubuntu SMP Thu Mar 28 22:45:47 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

user@fact:~$ psql -V
psql (PostgreSQL) 14.11 (Ubuntu 14.11-1.pgdg20.04+1)

user@fact:~$ ~/FACT_core/src/install.py
[2024-04-25 12:34:18][install][INFO]: FACT Installer 1.2
[2024-04-25 12:34:18][common][INFO]: Updating system
[2024-04-25 12:34:21][common][WARNING]: FACT is not set up using git. Note that *adding submodules* won't work!!
[2024-04-25 12:34:21][install][INFO]: Installing apt-transport-https autoconf automake build-essential git libtool python3 python3-dev unzip libfuzzy-dev libmagic-dev
[2024-04-25 12:34:22][common][INFO]: Installing python3 pip
[2024-04-25 12:35:13][install][INFO]: Installing nodejs papirus-icon-theme
[2024-04-25 12:35:43][frontend][INFO]: Creating directory for authentication
[2024-04-25 12:35:43][frontend][INFO]: Initializing docker container for radare
[2024-04-25 12:35:44][frontend][INFO]: Pulling pdf report container
[2024-04-25 12:35:45][db][INFO]: Setting up PostgreSQL database
OK
Hit:1 http://de.archive.ubuntu.com/ubuntu focal InRelease
Hit:2 http://de.archive.ubuntu.com/ubuntu focal-updates InRelease                      
Hit:3 http://de.archive.ubuntu.com/ubuntu focal-backports InRelease                    
Hit:4 http://de.archive.ubuntu.com/ubuntu focal-security InRelease                     
Hit:5 https://download.docker.com/linux/ubuntu focal InRelease                                                     
Hit:6 https://deb.nodesource.com/node_20.x nodistro InRelease                                                      
Hit:7 http://apt.postgresql.org/pub/repos/apt focal-pgdg InRelease         
Reading package lists... Done
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following packages were automatically installed and are no longer required:
  bridge-utils python3-cached-property python3-docker python3-dockerpty python3-docopt python3-texttable
  python3-websocket ubuntu-fan wmdocker
Use 'sudo apt autoremove' to remove them.
The following additional packages will be installed:
  postgresql-client-14 postgresql-client-common postgresql-common
Suggested packages:
  postgresql-doc-14
The following NEW packages will be installed:
  postgresql-14 postgresql-client-14 postgresql-client-common postgresql-common
0 upgraded, 4 newly installed, 0 to remove and 0 not upgraded.
Need to get 0 B/17.8 MB of archives.
After this operation, 60.1 MB of additional disk space will be used.
Preconfiguring packages ...
Selecting previously unselected package postgresql-client-common.
(Reading database ... 170678 files and directories currently installed.)
Preparing to unpack .../postgresql-client-common_259.pgdg20.04+1_all.deb ...
Unpacking postgresql-client-common (259.pgdg20.04+1) ...
Selecting previously unselected package postgresql-common.
Preparing to unpack .../postgresql-common_259.pgdg20.04+1_all.deb ...
Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common'
Unpacking postgresql-common (259.pgdg20.04+1) ...
Selecting previously unselected package postgresql-client-14.
Preparing to unpack .../postgresql-client-14_14.11-1.pgdg20.04+1_amd64.deb ...
Unpacking postgresql-client-14 (14.11-1.pgdg20.04+1) ...
Selecting previously unselected package postgresql-14.
Preparing to unpack .../postgresql-14_14.11-1.pgdg20.04+1_amd64.deb ...
Unpacking postgresql-14 (14.11-1.pgdg20.04+1) ...
Setting up postgresql-client-common (259.pgdg20.04+1) ...
Setting up postgresql-client-14 (14.11-1.pgdg20.04+1) ...
update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode
Setting up postgresql-common (259.pgdg20.04+1) ...

Creating config file /etc/postgresql-common/createcluster.conf with new version
Building PostgreSQL dictionaries from installed myspell/hunspell packages...
Removing obsolete dictionary files:
'/etc/apt/trusted.gpg.d/apt.postgresql.org.gpg' -> '/usr/share/postgresql-common/pgdg/apt.postgresql.org.gpg'
Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service → /lib/systemd/system/postgresql.service.
Setting up postgresql-14 (14.11-1.pgdg20.04+1) ...
Creating new PostgreSQL cluster 14/main ...
/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.

The database cluster will be initialized with locale "en_US.UTF-8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".

Data page checksums are disabled.

fixing permissions on existing directory /var/lib/postgresql/14/main ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
Processing triggers for man-db (2.9.1-1) ...
Processing triggers for systemd (245.4-4ubuntu3.23) ...
[2024-04-25 12:36:15][db][INFO]: Initializing PostgreSQL database
[2024-04-25 12:36:17][install][INFO]: Installing libjpeg-dev libssl-dev redis binutils file openssl bison flex pkg-config
[2024-04-25 12:36:29][install][WARNING]: Pip package pyyaml==6.0.1 is already installed with distutils. This may Cause problems:
Collecting pyyaml==6.0.1
  Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (2.1 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Installing collected packages: pyyaml
  Attempting uninstall: pyyaml
    Found existing installation: PyYAML 5.3.1
ERROR: Cannot uninstall 'PyYAML'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.

[2024-04-25 12:36:33][backend][INFO]: Skipping yara installation: Already installed and up to date
[2024-04-25 12:36:33][backend][INFO]: Installing checksec.sh
[2024-04-25 12:36:33][backend][INFO]: Pulling fact extraction container
[2024-04-25 12:36:34][backend][INFO]: Installing cwe_checker plugin.
[2024-04-25 12:36:35][backend][INFO]: Finished installing cwe_checker plugin.

[2024-04-25 12:36:35][backend][INFO]: Installing users_and_passwords plugin.
[2024-04-25 12:36:39][backend][INFO]: Finished installing users_and_passwords plugin.

[2024-04-25 12:36:39][backend][INFO]: Installing known_vulnerabilities plugin.
[2024-04-25 12:36:40][backend][INFO]: Finished installing known_vulnerabilities plugin.

[2024-04-25 12:36:40][backend][INFO]: Installing linter plugin.
[2024-04-25 12:36:58][backend][INFO]: Finished installing linter plugin.

[2024-04-25 12:36:58][backend][INFO]: Installing ipc plugin.
[2024-04-25 12:36:59][backend][INFO]: Finished installing ipc plugin.

[2024-04-25 12:36:59][backend][INFO]: Installing crypto_hints plugin.
[2024-04-25 12:36:59][backend][INFO]: Finished installing crypto_hints plugin.

[2024-04-25 12:36:59][backend][INFO]: Installing kernel_config plugin.
[2024-04-25 12:37:03][backend][INFO]: Finished installing kernel_config plugin.

[2024-04-25 12:37:03][backend][INFO]: Installing input_vectors plugin.
[2024-04-25 12:37:04][backend][INFO]: Finished installing input_vectors plugin.

[2024-04-25 12:37:04][backend][INFO]: Installing binwalk plugin.
[2024-04-25 12:37:15][backend][INFO]: Finished installing binwalk plugin.

[2024-04-25 12:37:15][backend][INFO]: Installing cve_lookup plugin.
[2024-04-25 12:38:28][backend][INFO]: Finished installing cve_lookup plugin.

[2024-04-25 12:38:28][backend][INFO]: Installing file_system_metadata plugin.
[2024-04-25 12:38:29][backend][INFO]: Finished installing file_system_metadata plugin.

[2024-04-25 12:38:29][backend][INFO]: Installing software_components plugin.
[2024-04-25 12:38:29][backend][INFO]: Finished installing software_components plugin.

[2024-04-25 12:38:30][backend][INFO]: Installing architecture_detection plugin.
[2024-04-25 12:38:32][backend][INFO]: Finished installing architecture_detection plugin.

[2024-04-25 12:38:32][backend][INFO]: Installing ip_and_uri_finder plugin.
[2024-04-25 12:38:39][backend][INFO]: Finished installing ip_and_uri_finder plugin.

[2024-04-25 12:38:39][backend][INFO]: Installing device_tree plugin.
[2024-04-25 12:38:41][backend][INFO]: Finished installing device_tree plugin.

[2024-04-25 12:38:41][backend][INFO]: Installing qemu_exec plugin.
[2024-04-25 12:38:42][backend][INFO]: Finished installing qemu_exec plugin.

[2024-04-25 12:38:42][backend][INFO]: Creating firmware directory
Create signature directory /home/user/FACT_core/src/analysis/signatures
Compile signatures in /home/user/FACT_core/src/plugins/analysis/known_vulnerabilities/signatures
Compile signatures in /home/user/FACT_core/src/plugins/analysis/crypto_material/signatures
Compile signatures in /home/user/FACT_core/src/plugins/analysis/crypto_hints/signatures
Compile signatures in /home/user/FACT_core/src/plugins/analysis/software_components/signatures
[2024-04-25 12:38:42][install][INFO]: installation complete


user@fact:~$ ~/FACT_core/start_all_installed_fact_components
[2024-04-25 12:39:45][start_all_installed_fact_components][INFO]: starting database
[2024-04-25 12:39:45][start_all_installed_fact_components][INFO]: starting frontend
[2024-04-25 12:39:45][start_all_installed_fact_components][INFO]: starting backend
[2024-04-25 12:39:46][migration][INFO]: Context impl PostgresqlImpl.
[2024-04-25 12:39:46][migration][INFO]: Will assume transactional DDL.
[2024-04-25 12:39:46][fact_base][ERROR]: The database schema in "storage/schema.py" does not match the schema of the configured database. Please run `alembic upgrade head` from `src` to update the schema.
[2024-04-25 12:39:46][migration][INFO]: Context impl PostgresqlImpl.
[2024-04-25 12:39:46][migration][INFO]: Will assume transactional DDL.
[2024-04-25 12:39:46][fact_base][ERROR]: The database schema in "storage/schema.py" does not match the schema of the configured database. Please run `alembic upgrade head` from `src` to update the schema.
Traceback (most recent call last):
  File "/home/user/FACT_core/src/../start_fact_database", line 51, in <module>
    FactDb().main()
  File "/home/user/FACT_core/src/../start_fact_database", line 38, in __init__
    super().__init__()
  File "/home/user/FACT_core/src/fact_base.py", line 40, in __init__
    self.do_self_test()
  File "/home/user/FACT_core/src/fact_base.py", line 88, in do_self_test
    raise DbInterfaceError('Schema mismatch')
storage.db_interface_base.DbInterfaceError: Schema mismatch
Traceback (most recent call last):
  File "/home/user/FACT_core/src/../start_fact_frontend", line 88, in <module>
    FactFrontend().main()
  File "/home/user/FACT_core/src/../start_fact_frontend", line 65, in __init__
    super().__init__()
  File "/home/user/FACT_core/src/fact_base.py", line 40, in __init__
    self.do_self_test()
  File "/home/user/FACT_core/src/fact_base.py", line 88, in do_self_test
    raise DbInterfaceError('Schema mismatch')
storage.db_interface_base.DbInterfaceError: Schema mismatch
[2024-04-25 12:39:46][migration][INFO]: Context impl PostgresqlImpl.
[2024-04-25 12:39:46][migration][INFO]: Will assume transactional DDL.
[2024-04-25 12:39:46][fact_base][ERROR]: The database schema in "storage/schema.py" does not match the schema of the configured database. Please run `alembic upgrade head` from `src` to update the schema.
Traceback (most recent call last):
  File "/home/user/FACT_core/src/../start_fact_backend", line 125, in <module>
    FactBackend().main()
  File "/home/user/FACT_core/src/../start_fact_backend", line 48, in __init__
    super().__init__()
  File "/home/user/FACT_core/src/fact_base.py", line 40, in __init__
    self.do_self_test()
  File "/home/user/FACT_core/src/fact_base.py", line 88, in do_self_test
    raise DbInterfaceError('Schema mismatch')
storage.db_interface_base.DbInterfaceError: Schema mismatch

@jstucke
Copy link
Collaborator

jstucke commented Apr 26, 2024

@SpaceMoehre are you using FACT 4.1.1 or the master branch?

@lrcarp17
Copy link

I had this issue, it seems like running DROP TABLE alembic_version; and re-installing fixed it for me

@lrcarp17
Copy link

However now it seems like I'm getting a 500 error when accessing firmware because the schema has changed. So idk if it's a true fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@jstucke @lrcarp17 @SpaceMoehre @maringuu