How to approach this complex report?

[David H.]

Hi all,

I'm fairly new to GRW and GRE and have managed to create several reports now without too many issues. My next challenge is a bit more complex so I'm wondering how best to approach this.

Basically its a report which requires multiple copies with front and back pages which vary per copy.

Copy 1) Has a front page containing multiple rows of data (ON EVERY ROW) and a blank back page.
Copy 2) Has a front page containing multiple rows of data (ON EVERY ROW) and a back page repeating much of that front data in a different layout with additional headings and a footer etc
Copy 3) As 2) but with a different title.

Any thoughts on how best to code and lay this out? One approach would be two separate reports/4rp files (each called 3x) one for the front pages and the other for the back pages, but these would then need to combined 3x somehow as page1 front page report, page 1 back page report, page2 front page report. page2 back page report, .... etc to make the final output

Any thoughts/suggestions?

Thanks,

David

[Alex G.]

Hi David,

I strongly suspect that it is not feasible to produce this report in one pass. In case that it isn't, you should look at the API functions fgl_report_setProcessLevelDataFile() and fgl_report_runReportFromProcessLevelDataFile() explained here https://4js.com/online_documentation/fjs-gst-manual-html/#gst-topics/c_grw_outputoptions_006.html and here https://4js.com/online_documentation/fjs-gst-manual-html/#gst-topics/c_grw_BDLfile_008.html#c_grw_BDLfile_008.
Using these API calls you can ensure that the database data used in the copies is the same as the data used in the original report even if the data in the data base changed between the runs.

Regarding "Copy 1" (backside of every page is a blank page) I see two options.
1) Turn backside printing off (usually this is the default setting. If it isn't then you can look at the API calls fgl_report_setPrinterSides() in the case that you are printing silently (without printer dialog) from the server).
2) Specify a blank backside as explained here https://4js.com/online_documentation/fjs-gst-manual-html/#gst-topics/t_grd_print_reverse.html.

Regarding "Copy 2" the solution depends on whether or not data can spill over from one page to another. The reason is that with standard backside printing the odd pages are the front side and the even pages are the back side so that spilling over would destroy the order. The Report Writer supports backside printing but the content of the backside needs to be constructed before the first row of data is printed on the front side and it can't exceed 1 page. In other words, it can't change from page to page based on the front page data. If however the data would be shipped in chunks that don't exceed the length of a page, and each chunk would be shipped twice (once for the front and once for the back side) then it would be feasible. However we generally consider it a bad idea when the server is aware of page breaks but in an application where it is clear that data can't exceed a page it wouldn't be problematic.
In the case that the report data can spill over multiple pages but it is ensured that the front pages and their corresponding back pages contain exactly the same rows (which is what I understand to be the case) then you might be able to find a command line utillity (most likely Ghoscript based) that offers the option to stitch two Postcript or PDF files by interleaving every other page from another file. A quick Google search yielded the names of the command line tools pdfseparate and pdfunite from the popper-utils package.

Regarding the setting of a different title in "Copy 3" you might want to look at the API function fgl_report_setEnvironment (https://4js.com/online_documentation/fjs-gst-manual-html/#gst-topics/c_grw_fgl_report_setEnvironment.html) and the RTL function Runtime.getEnvironmentVariable(). With these you can add variables to your report that are not part of the data stream so that your data stream is the same for the original and the copies but you still can provide data via the API that is different between two runs.

Best regards,

Alex

[David H.]

Thanks Alex, Yes the number of rows on the back pages will be the same and will very likely exceed a page. TBH its a bit disappointing to have to have to resort to looking into 3rd party tools to stitch files back together. One of the reasons for looking at GRW/GRE now was to try to avoid 3rd party tools, like PDFLib, which is what we used previously for PDF output. If there is no built in way to achieve this then it would seem a good idea to have the facility to combine report output files in GRE/GRW, preferably supporting options to append and interleave (page by page) etc.

Regards,

David

[David H.]

One further thing. If for copies 2 and 3 I'm creating 2 reports each for the front/back pages and combining them somehow, what would you suggest is the best way to make the 1st file Pages numbered 1,3,5,7 of X etc and the 2nd file Page 2,4,6,8 of X, where X is the total number of pages of both copies combined?

TIA,

David

[Alex G.]

Hi David,

You could proceed as follows:
First you run one report that produces the front sides using the design file FrontSides.4rp. The data for this run comes from the database. The run is configured to output a pdf file (say FrontSides.pdf) and a xml data file configured by the API call fgl_report_setProcessLevelDataFile() as explained earlier.
When the report has finished you obtain the total number of pages produced by calling the API function fgl_report_getTotalNumberOfPages(). This information will be needed later.
Then you run a second report that produces the back sides using the design file BackSides.4rp. The data for this run comes from the data file produces in the first run by using the API call fgl_report_runFromProcessLevelDataFile(). The run is configured to output a pdf file (say BackSides.pdf).
You could call the function fgl_report_getTotalNumberOfPages() to verify that the two report have actually produces the same number of pages.
In a third step you make a report design FrontAndBackSide.4rp that contains two pages inside the OnEveryRow trigger (it is a bit unusual but totally legal to do).
This report should have no paper margins configured.
In each of the pages you place a PDFBox. The location of the first one points to FrontSides.pdf and the other one to BackSides.pdf. The "Page Ranges" attribute is set to the RTL expression pageRanges="{page_number}".
The variable page_number comes from a 4GL report that prints in ON EVERY ROW a single INTEGER variable page_number that is incremented on every call to OUTPUT TO REPORT.
OUTPUT TO REPORT is called in a FOR loop from 1 to the total number of pages obtained in the first run above.

If you don't shy back to manipulated XML documents then you can also look at the SVG that we produce. In those files we use "PageSet" and "Page" tags to separate pages. Putting together a new file that consists of pages that were grabbed from two other files is easily doable in 4GL using SAX or DOM. This is probably more efficient than the solution sketched out above but limits the output format to SVG. 
Yes, we could offer some sort of merging mechanism in a future version. I will see that a request for enhancement for this is registered.

Best regards,

Alex   

[Bryce S.]

Hi David,

This all seems rather complicated but maybe I'm missing some of the nuances of your requirement.
We have some fairly complex reports where pages are printed or not depending on flags and values passed to the report.  One I just looked up makes a lot of use of the visibility conditions for the various pages to decide whether to show them using RTL like for example back_page_need_blank==1 or (!temp_table_record.tmp_licence.trim().isEmpty()&&temp_table_record.tmp_licence!="S"&&temp_table_record.tmp_age>=65)||(report_type=="A"&&temp_table_record.tmp_age>=65&&show_starters!=1)

If you need the blank back pages for double sided printing, this report does that too with flags if you know it must be blank, or with visibility condition testing the count of records printed I know how many fit on a page, if it exceeds value it uses a data page for the back side, if not it uses a 'blank page' instead (there may be a better way to tell if data flowed over to the next page).

So you could get your alternative reports out by just running it again with different flags to say what you want? For reprinting some reports we generate the data that is being sent to the report and save it to the database first, then print and/or reprint is done from this data so reports are consistent.

Maybe something like this fits your situation?
Regards, Bryce.

[David H.]

Thanks Alex. I was hoping it might be a bit simpler than that, something like being able to specify the start page number and page number increment for the PageNoBox, so for the front pages I could initialize it with 1=start,2=increment and the back page 2=start,2=increment. Maybe this could be a useful enhancement for a future version?

Thanks also to Bryce. Yes I became aware of visibility conditions last week (this is only my 3rd report in GRE/GRW) and planned to use them to alter the output for the front and back pages. What I don't really understand it the stuff relating to pages and how many rows fit on a page enough yet and how to detect if content has overflowed a page etc. For 2 of the copies this report needs, rows output will most likely exceed a single page and if X rows are printed on the front page some of the information of each of those same X rows needs to be printed in a different format on the back page.

Regards,

David

[Alex G.]

Hi David,

I thought that I had understood the requirement but with your latest reply I am not so sure anymore.
I was concentrating on the request "Copy 2" that you formulated as follows:

>Copy 2) Has a front page containing multiple rows of data (ON EVERY ROW) and a back page repeating much of that front data in a different layout with additional headings and a footer etc

You confirmed that a) the number of rows that fit on a front page is unknown and b) that the data can spill over to other front pages.

If I reword the request in my words it would be this:

I have a multi-pages report that prints a list of data from ON EVERY ROW (Report 1). I have another report that prints the same data in a different layout (Report 2) but it is guaranteed that both reports have the same number of pages and that the data of individual pages is the same (they are just formatted differently)
I want to obtain a third report (Copy 2) that contains all the pages from both reports in such a way that page 1 is page 1 from Report 1, page 2 is page 1 from Report 2, page 3 is page 2 from Report 1, page 4 is page 2 from Report 2 and page 2*n+1 is page n from report 1 and page 2*n is page n from report 2.

However in your latest reply you wrote:

>I was hoping it might be a bit simpler than that, something like being able to specify the start page number and page number increment for the PageNoBox, so for the front pages I could initialize it with 1=start,2=increment and the back page 2=start,2=increment. Maybe this could be a useful enhancement for a future version

I have don't understand how that relates or would solve the Copy 2 problem.
Also I don't understand why you would want to start and increment the way you describe. If you initialize the first page with 1 and then increment by 2 then the front sides will be numbered 1,2,3,5,7,.... Likewise numbering the backsides starting with 2 and incrementing by 2 yields 2,4,6,8,.. so that the numbering of all the pages (alternating front and back pages) will be 1,2,3,4,5,6,8...
That on the other hand you can achieve by just taking a PageNoBox as is and it will give you this numbering.
We do support different start values (we have the concept of logical and physical pages) that might possibly helpful but I feel that I first need to fully understand what you are trying to achieve.
Is it perhaps possible that you provide an example document?

[David H.]

Hi Alex,

Apologies for the confusion and yes you did understand the requirement perfectly I think initially.

I have confused you as I don't understand how page numbering works I guess. When I said for the front page report start with 1 and increment by 2, I was expecting this would give me 1, 3, 5, 7 etc 1, 1+2, 1+2+2, 1+2+2+2 and on the back page report 2,4,6,8 as 2, 2+2, 2+2+2, 2+2+2+2. The two PDF reports when combined by a merge tool would then give me the correct output, i.e. 1,2,3,4,5,6,7,8... I don't see how you get 1,2,3,5,7 for the front page report!

[Alex G.]

Hi David,

Now I understand and yes, that is a valid concern. For getting the page numbering correct in the final document I propose to simply overpaint the page numbers by putting two PageNoBoxes in the combined report (FrontAndBackside.4rp).

I have just now tested the complete approach. In the 4GL I have:

MAIN
    LET handler ..
    CALL runNumberUpReport(handler,3)
END MAIN

FUNCTION runNumberUpReport(handler,numberOfPages)
    DEFINE handler om.SaxDocumentHandler,
           i,numberOfPages INTEGER
    START REPORT NumberUpReport  TO XML HANDLER HANDLER
    FOR i=1 to numberOfPages
        OUTPUT TO REPORT NumberUpReport(i)
    END FOR
    FINISH REPORT NumberUpReport
END FUNCTION

REPORT NumberUpReport( i )
    DEFINE i INTEGER
FORMAT
    ON EVERY ROW
        PRINT i
END REPORT

I created a design file as described earlier with two page roots, two PDFBoxes and two PageNoBoxes and called it CombinedReport.4rp (see attached)

For input I used the OrderReportDemo report to produce two PDF files named as their respective .4rp files GroupedTableDemo.pdf and OrderList.pdf.
This method works for any two pdf files. The names of the files are statically set in the report design but they could also be set from a variable in the 4GL.

Note that the resulting pdf prints fine but the file is a lot larger in size than the sum of the two input files. The process strips embedded fonts from the original input pdf's and draws text by using outlines which inflates the file. The quality is not degraded by this so that also high resolution printout will be OK.
Please not that I didn't pay attention that the two reports show the same data on the same pages. I just took the pdf output as is. Actually, one report had a total of 4 pages and the other one 3 so that I just took the first three pages.

I know that you said that you found it too complicated and perhaps it is, but since I tried it out, I thought that I might as well share it with you.

Best Regards,

Alex

[David H.]

Thanks Alex, I'll give that a go. Hopefully the increased size won't cause me an issue.

If currently have one 4rp file with 2 environment variables copy (1,2 or 3) and front=(0,1) and an evenPageHeader (size x=max.y=max, with visibility copy=1 for a blank page) In this report I have also 2 headers and 2 table row sections set the visibility as follows:-

FrontPageHeader / FrontPageTableRow = copy=="1" || front=="1"
BackPageHeader /BackPageTableRow = copy!="1" && front =="0"

This seems to work and I can then call it as follows to get the various PDF outputs I need:-

copy=1 and front = 1 (Front pages of copy 1 each with a blank back page)
copy=2 and front = 1 (Front page copy 2)
copy=2 and front = 0 (Back page copy 2)
copy=3 and front = 1 (Front page copy 3)
copy=3 and front = 0 (Back page copy 3)

So all the main pieces are there now and hopefully I can just concentrate now on making it look good!

Regards,

David

[Reuben B.]

IF and it is a big IF, if the amount of vertical space to display each row is the same e.g. 1cm, and the amount of vertical space to display the header/footer is the same e.g. 5cm, then I wonder if you could achieve it with a single 4rp file and duplicate the data in the data-stream. 

This relies on fact if page size is fixed e.g. A4 = 29.7cm, header/footer is fixed e.g. 5cm, row size is fixed e.g. 1cm, then you know there is room to display 24 rows on each page.

So your 4gl would have a page_number and a row_number variable, and it is the 4gl responsibility to increment the row_number and to set row_number back to 1 and increment page_number every 24 rows, and to resend same data to duplicate the pages.  Your 4gl report then has

BEFORE GROUP OF page_number
   PRINT page_number, ...
BEFORE GROUP OF row_number
   PRINT row_number
ON EVERY ROW
   PRINT row_data.*

CONSTANT page_size = 24


and your 4rp designer can place Page_root under page_number, and look to see if page_number is odd or even to determine if any printing occurs on back page.

This approach only works if the maximum number of rows that can print on the page is fixed and known and you can use that in the 4gl.  If the row or page header/footer height is not fixed you cannot use this technique.  If something happens to change the number of rows that can fit on the page i.e report designer decides a 12pt instead of 10pt font is required, changes the image size in header, change portrait to landscape, then the number in the 4gl needs to be altered to reflect the number of rows that will fit on the page. 

Normally you would let the report layoutter manage the pages but using this technique you can effectively control the page throws from the 4gl.

Reuben

[David H.]

Thanks Reuben. Despite its downsides I gave this a go and now have all 3 copies coming out of 1 report and a single XML data stream. I preferred this approach because it's then no different to any other reports we generate and I did not have to.do all the messing around to get page numbering right or combining multiple PDF documents to make my final output.