How do seed words influence the topic outputs?

[sruti]

Thanks for this great package! I am trying to use seeded BERTopic to find certain climate topics of interest in documents. The seed words have been compiled with knowledge from domain experts and have gone through several iterations. However, I am noticing that, keeping everything else the same, even with relatively small changes to the seed words the topic outputs look fairly different. For example, "renewable energy" might be be the 3rd topic in one iteration and the 7th topic on the next run (with slightly different seed words) even though the change in seed words had nothing to do with "renewable energy". I have ~60 seed topics and I am getting 500-600 topics in the output.

Do you have insight into why this may be or how to evaluate how changing the seed words directly changes the output? From my understanding, seeded topic modeling only affects the topic representation and not topic assignment. So I would imagine atleast the top ~15 topics to not change, with perhaps different words to describe the topics. But I am seeing the order of the top 15 topics changing and some topics appearing/disappearing when modifying the seed words.

To put it another way, how do I know "renewable energy" is the 3rd topic because that's the topic 3rd most discussed vs being 3rd because there are more seed words for "renewable energy" so the topic is oversampled?

Here is the code:

# Generate embeddings for each document
embedding_model = SentenceTransformer('all-MiniLM-L6-v2')
embeddings = embedding_model.encode(sentences, convert_to_tensor=False, show_progress_bar=True)

## Seeded topic modeling with BERT 
vectorizer_model = CountVectorizer(stop_words="english", ngram_range=(1,3))

representation_model = KeyBERTInspired()

#Set random seed for umap for reproducibility
umap_model = UMAP(random_state=42)

# Initialize BERTopic with seed topics
topic_model = BERTopic(seed_topic_list=seed_topic_list, umap_model=umap_model, vectorizer_model=vectorizer_model, embedding_model=embedding_model, min_topic_size=50, representation_model=representation_model)

# Fit the BERTopic model on the embeddings
topics, probs = topic_model.fit_transform(sentences, embeddings)

I realize answering this question exactly without seeing the data is difficult but some insight into understanding the behavior would be very helpful. Thanks for your help and please let me know if more information would be helpful!

[MaartenGr]

Thank you for the kind words!

The thing you are referencing is not what you are using. That issue talks about seed_words in the ClassTfidfTransformer (which you are not using) and zero-shot topic modeling (which you are also not using).

The correct documentation for your functionality is the guided topic modeling. I would highly advise to check that out as it explains how it affects the underlying embeddings.