We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
osccap/osccap/agilent.py
Line 56 in efec4ee
need to calculate twos complement: ((255<<8)+255)-0x10000 * 1 = -1 smallest negative number: ((128<<8)+0)-0x10000 * 1 = -32768 biggest positive number: ((127<<8)+255)-0x10000 * 0 = 32767
addtitional parentheses needed
timeit.Timer('a=255;b=255;((a<<8)+b)-0x10000*(a<<7)', 'gc.enable()').timeit() 0.2427769000059925 timeit.Timer("a=255;b=255;int(format(a,'08b')+format(b,'08b'),2)", 'gc.enable()').timeit() 1.5376816670177504
words_to_int(high_byte, low_byte): if high_byte < 128: high_byte0x100+b else: high_byte0x100+b-0x10000
The text was updated successfully, but these errors were encountered:
No branches or pull requests
osccap/osccap/agilent.py
Line 56 in efec4ee
need to calculate twos complement:
((255<<8)+255)-0x10000 * 1 = -1
smallest negative number: ((128<<8)+0)-0x10000 * 1 = -32768
biggest positive number: ((127<<8)+255)-0x10000 * 0 = 32767
addtitional parentheses needed
words_to_int(high_byte, low_byte):
if high_byte < 128:
high_byte0x100+b
else:
high_byte0x100+b-0x10000
The text was updated successfully, but these errors were encountered: