We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Can I ask some questions? In the single_inference_7b_13b.py, there exists code in the implementation of SupervisedDataset class,
SupervisedDataset
self.input_ids = data_dict["input_ids"] + data_dict["input_ids"][-100:] self.labels = data_dict["labels"] + data_dict["labels"][-100:]
I don't know the reason of data_dict["input_ids"][-100:];
data_dict["input_ids"][-100:]
And here is the implementation of padding function:
padding
def padding(inputs, padding_token, cutoff = None): num_elems = len(inputs) if cutoff is None: cutoff = max([len(item) for item in inputs]) else: cutoff = min(max([len(item) for item in inputs]), cutoff) tokens = torch.ones(num_elems, cutoff).long().to(inputs[0].device) * padding_token for i in range(num_elems): toks = inputs[i] length = min(cutoff, len(toks)) tokens[i, -length:] = toks[-length:] return tokens
It seems that the side of padding is left, why here the side is left?
left
The text was updated successfully, but these errors were encountered:
No branches or pull requests
Can I ask some questions?
In the single_inference_7b_13b.py, there exists code in the implementation of
SupervisedDataset
class,I don't know the reason of
data_dict["input_ids"][-100:]
;And here is the implementation of
padding
function:It seems that the side of padding is
left
, why here the side isleft
?The text was updated successfully, but these errors were encountered: