update kt-kernel

[ovowei]

No description provided.

[gemini-code-assist]

Summary of Changes

Hello @ovowei, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the development workflow by introducing automated code quality checks via Git hooks and streamlining dependency management. Crucially, it integrates high-performance CPU kernels from the llamafile project, which are essential for optimizing matrix multiplication and Mixture of Experts operations, thereby improving the overall computational efficiency of the kt-kernel project.

Highlights

• Git Hooks for Code Quality: Implemented commit-msg and pre-commit Git hooks to enforce Conventional Commits and automatically format code using clang-format and Black.
• Submodule Integration: Added pybind11 and llama.cpp as Git submodules to manage external dependencies.
• Dependency Management Refinement: Updated pyproject.toml to clearly separate core and development dependencies, and introduced requirements.txt for optional pre-installation.
• Optimized CPU Kernels: Integrated a significant portion of the llamafile project, providing highly optimized CPU matrix multiplication (tinyBLAS) and Mixture of Experts (MoE) kernels for various architectures (AVX, AVX2, AVX512, ARM NEON/Dotprod).

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces kt-kernel, including build infrastructure, dependencies, git hooks, and a significant amount of performance-oriented C++ code adapted from llamafile for CPU-based kernel operations. The changes are extensive and set up a new component in the repository.

My review focuses on the new scripts and the general structure of the added C++ code. I've identified a few issues:

• A logic error in the commit-msg git hook that would prevent developers from using the documented commit message format for scopes.
• A robustness issue in the pre-commit hook when handling Python files with spaces in their names.
• A recurring typo in copyright notices across many of the new C++ files.
• Formatting problems in function declarations within sgemm.h that affect readability.

Overall, the additions are substantial. Addressing these points will improve the developer experience and code quality.

[gemini-code-assist]

The commit message format description and examples are inconsistent with the validation regex.

1. The format description on line 41 is missing the optional breaking change indicator !.
2. The example on line 45 for a commit with a scope, [fix(parser)]: ..., will be rejected by the regex, which expects the scope to be outside the square brackets, like [fix](parser): ....

To ensure the hook works as intended and the guidance is clear, both the description and the example should be aligned with the regex.

Suggested change

	[type](scope)?: subject
	Examples:
	[feat]: add new feature
	[fix(parser)]: handle edge case
	[type](scope)?[!]?: subject
	Examples:
	[feat]: add new feature
	[fix](parser): handle edge case

[gemini-code-assist]

The current method of passing python files to black is not robust. It will fail for filenames containing spaces due to word splitting, and it could potentially exceed the shell's argument length limit if many files are staged. Using mapfile (a bash 4+ feature) to read files into an array is a more robust approach that correctly handles spaces in filenames.

Suggested change

	PY_FILES=$(git diff --cached --name-only --diff-filter=ACM | grep -E '\.py$' || true)
	if [ -n "$PY_FILES" ]; then
	echo "[pre-commit] running black on staged python files..." >&2
	$BLACK_BIN $PY_FILES
	else
	# Optionally format all python files; comment out if not desired
	# $BLACK_BIN "$REPO_ROOT"
	:
	fi
	mapfile -t PY_FILES < <(git diff --cached --name-only --diff-filter=ACM | grep -E '\.py$' || true)
	if [ ${#PY_FILES[@]} -gt 0 ]; then
	echo "[pre-commit] running black on staged python files..." >&2
	"$BLACK_BIN" "${PY_FILES[@]}"
	fi

[gemini-code-assist]

There is a typo in the word "Copyright". This typo appears in the copyright notices of multiple newly added files from third_party/llamafile/.

Suggested change

	// Copyrigth 2024 Mozilla Foundation.
	// Copyright 2024 Mozilla Foundation.

[gemini-code-assist]

The function declarations in this block have formatting issues, with missing spaces between types and some parameter names (e.g., long,int,). This harms readability and could be problematic for some tools or compilers. It's recommended to format them consistently.

Suggested change

	bool iqk_mul_mat(long, long, long,int, const void*, long, int, const void*, long,float*, long, int, int);
	bool iqk_mul_mat_zen4(long, long, long,int, const void*, long, int, const void*, long,float*, long, int, int);
	bool iqk_mul_mat_arm82(long, long, long,int, const void*, long, int, const void*, long,float*, long, int, int);
	bool iqk_mul_mat(long, long, long, int, const void*, long, int, const void*, long, float*, long, int, int);
	bool iqk_mul_mat_zen4(long, long, long, int, const void*, long, int, const void*, long, float*, long, int, int);
	bool iqk_mul_mat_arm82(long, long, long, int, const void*, long, int, const void*, long, float*, long, int, int);