IBM PTFs required for correct maths calculations (precision) when using RDMLX on iSeries

Date:	10 August 2007
Product/Version:	Visual LANSA (RDMLX) on iSeries
Abstract:	Incorrect precision calculations when using RDMLX on iSeries fixed via PTF
Submitted By:	LANSA Technical Support
External(s):	IBM PTFs
Last Review:	November 2010

In Visual LANSA 11.x, it is possible to create RDMLX functions to execute on iSeries. During checkin and compile, C code is generated using the IBM C compiler.

In some cases the ILE C compiler (CRTCMOD and CRTBNDC commands) incorrectly calculates intermediate results for packed decimal division. There is available from IBM a PTF to correct this problem. If you experience any calculation anomalies, you should apply the appropriate PTF, depending on your OS version.

PTF number for V5R3 is SI27397
PTF number for V5R4 is SI27458

Sample code to illustrate the problem:

define field(#fielda) type(*dec) length(15) decimals(4) edit_code(J)

            define field(#fieldb) type(*dec) length(15) decimals(0) edit_code(J) default(6000)

            define field(#fieldc) type(*dec) length(15) decimals(4) edit_code(J) default(6000)

            

            #fielda := #fieldb / 10000  or change field(#fielda) to('#fieldb / 10000')

            results in #fielda = 0.0000    incorrect

            

            #fielda := #fieldc / 10000 or change field(#fielda) to('#fieldc / 10000')

            results in #fielda = 0.6000    correct

With the PTF applied, both answers are 0.6000. Refer to the PTF document for a full explanation.