Python and integer representation (leading zeros issue)



Question:

This is similar to a question asked on the programming Stack Exchange: http://programmers.stackexchange.com/questions/158247/binary-representation-in-python-and-keeping-leading-zeros
Essentially, I have some numbers that I keep track of in hex:
SOME_NUMBERS = (0xAABBCCDDEEFF, 0xAA55AA55AA55, 0xBEEF0000BEEF)  
However, Python tries to outsmart me and over-thinks, so when I write the value to a file, I get this in my hex editor:
00 00 00 00 00 00 00 00 00 00 AA BB CC DD EE FF  
It looks to me that Python stores and writes my 48-bit number as a 64- or 128-bit number. How could I prevent the leading zeroes from being written to the file? By comparison, when using numpy.random.bytes(6), Python will write the values to the file without leading zeroes, so it should be capable of doing this I think. Thoughts?


Solution:2

The critical thing is how you are packing the numbers. You can write single bytes - it is unusual for a number format to be 6 bytes. I found the following to work for one of the values:
import struct    outfile = 'output.dat'  fh = open(outfile, "wb")    var = struct.pack('6B', 0xAA, 0xBB, 0xCC, 0xDD, 0xEE, 0xFF)    fh.write(var)  fh.close()  
If you use other types, like a short, int, or long (or combinations thereof) you might run into endian issues.


Solution:3

If you want a 48-bit binary value written to a file, you'll have to write 6 bytes yourself. There's no native number format of that length. Use the struct module to convert a value to bytes:
packed = struct.pack('>q', value) # 64 bits (8 bytes)  f.write(packed[-6:])  


Solution:4

To fix my issue, I had to do two things:
1) Switch to Python 3 2) Use int.to_bytes()
f = open(fileName, 'wb')  for d in dataBuffer:      if type(d) == int:           f.write(d.to_bytes(6, byteorder='big'))      else:           f.write(d)      f.close()  
I couldn't use struct.pack() or the numpy.savetext(), as I needed to handle 10,000 random values for 2000 files. This seemed easier, but I also had to make my code work with Python 3. I've learned that Python, while trying to be smart and do things for the user, ends up making a mess of things and increasing frustration levels.


Solution:5

You could save as text using np.savetxt:
import numpy as np  sn = np.array([0xAABBCCDDEEFF, 0xAA55AA55AA55, 0xBEEF0000BEEF])  np.savetxt('testfile',sn,'0x%X')  
or
np.savetxt('testfile',sn,'%X') # without the '0x'  


Solution:6

If you don't want to use struct.pack, you can write the bytes individually to file by masking out successive bytes of the source integer and using chr to convert them to byte strings :
for shift in range(40, -1, -8):      f.write(chr((d >> shift) & 0xff))  
But this is basically what struct.pack is doing for you under the hood (and in C), so you'd be better off not reinventing the wheel.

Note:If u also have question or solution just comment us below or mail us on toontricks1994@gmail.com
Previous
Next Post »