ReadHistory throws "HistoryReadRawModified has bad status BadInternalError" when variable is written from external Client

[sricharan-code]

Hi All,

I've implemented opc ua server and enabled the variable with historizing using the below server code:
server.historize_node_data_change(var_object, period=Period, count=Count).

I'm able to write the data to the same variable with opc ua client using the below client code:
var_object.write_value(VarValue, DataTypeDict[VarType])

I'm able to read the history from client using the below client code:
var_object.read_raw_history(start_time,end_time, count)

All those operations are successful. However, when I connect the server with uaexpert & write the data to historized variable, History read fails with the error "HistoryReadRawModified has bad status BadInternalError"

I'm not able to understand the difference between writing from asyncua client(which works) and uaexpert(which throws error).

I've gone through the existing code & opc ua specifications, found that variable IsReadModified in read_raw_history differentiates reading raw history and modified history. However, the same variable is assigned as False in read_raw_history method present in node.py file. I'm not sure if it has any connection with the error I'm facing.

Please let me know what I'm missing here.

Regards,
Charan

[oroulet]

I have no ideas what the issue is, but if you use wireshark or similar you can know what exactly uaexpert is writting and compare to our client and hopefully you will spot a difference.
Also don't you et some stack trace on server side when that error happens?

[sricharan-code]

Thanks for the quick respose @oroulet . I'm able to find the rootcause for read history failure.
It is becase the SourceTimestamp isn't set while writing the data from UAExpert. Whereas in asyncua client's case, we are assigning SourceTimestamp in value_to_datavalue method.
Apparently, it links back to FreeOpcUa/python-opcua#727.
Any idea why SourceTimestamp is missing when writing from external client?

[oroulet]

btw. are you using opcua-asyncio as server? if you get InternalError you shoudl have a stack trace there. What does it say?
Also I think we do not write Sourcetimestamp by defualt. Write a DataValue instead of a vriant or python object and you can set SourceTimestamp on client side. I am sure there is an example in examples folder

[sricharan-code]

Yes. I'm using opcua-asyncio server. Below is the stack trace,

ERROR:asyncua.server.uaprocessor:Error while processing message
Traceback (most recent call last):

File "c:\Users\sricharan.a\AppData\Roaming\Python\Python39\site-packages\asyncua\server\uaprocessor.py", line 126, in process_message        
    return await self._process_message(typeid, requesthdr, seqhdr, body)

File "c:\Users\sricharan.a\AppData\Roaming\Python\Python39\site-packages\asyncua\server\uaprocessor.py", line 370, in _process_message       
    results = await self.session.history_read(params)
  
File "c:\Users\sricharan.a\AppData\Roaming\Python\Python39\site-packages\asyncua\server\internal_session.py", line 119, in history_read      
    return await self.iserver.history_manager.read_history(params)
  
File "c:\Users\sricharan.a\AppData\Roaming\Python\Python39\site-packages\asyncua\server\history.py", line 290, in read_history
    res = await self._read_history(params.HistoryReadDetails, rv)
  
File "c:\Users\sricharan.a\AppData\Roaming\Python\Python39\site-packages\asyncua\server\history.py", line 305, in _read_history
    dv, cont = await self._read_datavalue_history(rv, details)
  
File "c:\Users\sricharan.a\AppData\Roaming\Python\Python39\site-packages\asyncua\server\history.py", line 331, in _read_datavalue_history    
    dv, cont = await self.storage.read_node_history(rv.NodeId,
  
File "c:\Users\sricharan.a\AppData\Roaming\Python\Python39\site-packages\asyncua\server\history.py", line 129, in read_node_history
    results = [dv for dv in self._datachanges[node_id] if start <= dv.SourceTimestamp]
  
File "c:\Users\sricharan.a\AppData\Roaming\Python\Python39\site-packages\asyncua\server\history.py", line 129, in <listcomp>
    results = [dv for dv in self._datachanges[node_id] if start <= dv.SourceTimestamp]
TypeError: '<=' not supported between instances of 'datetime.datetime' and 'NoneType'

Also I think we do not write Sourcetimestamp by defualt. Write a DataValue instead of a vriant or python object and you can set SourceTimestamp on client side. I am sure there is an example in examples folder

I've found this working with opcua-asyncio client. However, with external client such as UAExpert, it itsn't getting updated.

Note: I'm sorry for the broken link in the previous comment. I was referring to FreeOpcUa/python-opcua#727.

[oroulet]

OK now it is clear what is happening here, it really looks like we should write timestamp by default....

[oroulet]

history does not make sense without timestamp

[oroulet]

or maybe we could use server timestamp if not timestamp exist... Not sure what spec says. Should have a look one day if I have time

[oroulet]

in the meatime write a DataValue with a sourcetimestamp and it will solve your issue

[sricharan-code]

Sure @oroulet . I'll assign it in Data value for time being. Thanks for the quick help.

[oroulet]

On fact I checked code and we do write Sourcetimestamp in client code! how did you manage not to write it?

[sricharan-code]

Source Timestamp is coming when writing it from opcua asyncio client. It didn't come when I write from uaexpert.

…

On Fri, Jun 18, 2021, 4:19 PM oroulet ***@***.***> wrote: On fact I checked code and we do write Sourcetimestamp in client code! how did you manage not to write it? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#608 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AQUSQPCMT5T24YIKP4L3QLDTTMQFDANCNFSM46ZOTMKA> .

[oroulet]

OK so it is uaexpert not writing source timestamp. Not sure we should force source timestamp on server side...

[sricharan-code]

I agree. The only doubt I have is either uaexpert isn't sending the timestamp or is timestamp getting lost when unpacking the struct from binary. Cause source timestamp plays an important role in historical access and I feel it's too good to miss from ua expert's end.