#501  
Old 14-01-2017, 09:07 AM
gehelem (Gilles)
Registered User

gehelem is offline
 
Join Date: Sep 2016
Location: Rambouillet - France
Posts: 112
Anybody tried new firmware/camview ?
Quick test gived no result for me (black images)
Need to investigate, but rolledback
Reply With Quote
  #502  
Old 14-01-2017, 12:19 PM
luka's Avatar
luka
Unregistered User

luka is offline
 
Join Date: Apr 2007
Location: Perth, Australia
Posts: 1,124
No issues here with the new firmware/camview. I even tried it with your indi driver last night and it worked fine.

Also there is no white line in the first row under kstars with the indi driver (linux).
Reply With Quote
  #503  
Old 14-01-2017, 07:51 PM
gehelem (Gilles)
Registered User

gehelem is offline
 
Join Date: Sep 2016
Location: Rambouillet - France
Posts: 112
Quote:
Originally Posted by luka View Post
No issues here with the new firmware/camview. I even tried it with your indi driver last night and it worked fine.

Also there is no white line in the first row under kstars with the indi driver (linux).
Ok cool
No issues with cooling under windows ?
Did you try in your living room or under real sky ?
> you would be the first to run my driver with real stars, i would really enjoy to see a picture, even a bad one
Reply With Quote
  #504  
Old 14-01-2017, 08:18 PM
luka's Avatar
luka
Unregistered User

luka is offline
 
Join Date: Apr 2007
Location: Perth, Australia
Posts: 1,124
I only have uncooled Cam86 in 3D printed (plastic housing), i.e. no cooling yet to test. But it can take images fine with the latest firmware.

I don't have linux on my laptop (the old laptop died so linux is only on my desktop) so I have to figure out some way to use the cam86 with linux outside the house. Do you know if the old Raspberry Pi 1 (model B, 256BM of RAM) can run indi? I have a few lying around.
Actually I may have a Beaglebone Black as well somewhere...

I also played with your driver today. I tried to get rid of the horizontal stripes by changing the FTDI parameters. Could not see any pattern that permanently removes them. I am not sure what is going on. I want to check if taking images with cam86_viewer results in horizontal lines.

I checked the Ukrainian forums and few people use MaximDL for acquisition.
Reply With Quote
  #505  
Old 14-01-2017, 11:15 PM
luka's Avatar
luka
Unregistered User

luka is offline
 
Join Date: Apr 2007
Location: Perth, Australia
Posts: 1,124
Gilles, got the driver working on Beaglebone Black. Few observations:
1. The lines are really bad, present in every image. Quite often it is not just lines but "blocks" of "misplaced" data showing vertical artifacts.

2. Acquisition is very slow. After the exposure it does
...
Try 2000 since 11778624<>12008000
for few seconds and then it takes few seconds again to transfer the image. I timed 15 seconds to take, transfer and show 1s exposure.
edit: for comparison taking and showing 1s exposure on my PC takes 5 seconds.

3. The CPU usage is 100% for several seconds. This seems to be part of the bottleneck. Only one CPU core is used at time. Firstly ./indi_cam86_ccd takes 100% of the CPU (image processing?) and then indiserver does (image transfer?).

Have you tried using the driver on a Raspberry Pi yet? The lines are present in almost every image on the BeableBone Black and they probably will be on Raspberry Pi as well. This may make it easier to tweak the FTDI parameters. I tried changing the parameters and at one stage got few good images but the same parameters did not work few minutes later. There is something weird going on.

Also what do lines like this mean?
Try 2000 since 11778624<>12008000
Reply With Quote
  #506  
Old 15-01-2017, 12:36 AM
gehelem (Gilles)
Registered User

gehelem is offline
 
Join Date: Sep 2016
Location: Rambouillet - France
Posts: 112
Quote:
Originally Posted by luka View Post
Gilles, got the driver working on Beaglebone Black. Few observations:
(...)
Also what do lines like this mean?
Try 2000 since 11778624<>12008000
Quick explanations time - be carefull i have flu, my head is "high in space"
There are two libraries we can use to talk to ft2232
  • Original D2XX from ftdi
  • LibFTDI from intra2net
The first one is the one used in ukrainian sofware, under windows.
I've first coded my cam84 driver with this one on linux. It is working quite well, no lines, no transmission troubles.
BUT : under linux, using bitbang mode with this library forces to unload ftdi_sio module.
The delicate consequence is that you can no more use other ftdi device, in a classical usb/serial converter for example. This is really annoying : can't use Eqmod on raspberry, nor any device using these devices (arduino/Moonlite focuser in my case)

The second library allows to keep module ftdi_sio module loaded, you "only" have to unbind it from kernel for a specific device (purpose of the udev file).
BUT : function calls are very similar, but there are a few specific items.
One of the biggest is the bitbang reading process. when you start a reading call, if there is nothing to read, function returns immediatly. This is the purpose of "ftdi_read_data_modified", wich is a hack submited by Toups on Indi forum :
http://www.indilib.org/forum/general...ver.html#12731
This function is reponsible of the messages you get ("Try 2000 since 11778624<>1200800").
What do they mean ?
1200080 is the size of the data buffer we try to recieve (twice the number of pixels, in fact)
The function is trying to wait 2000x1us (max) that buffer is fullfilled.
I also have often these messages : i'm not really sure, but i think it is because the reading process is not synchronized with the "rythm" given by atmega.
Here, with "Try 2000 since 11778624<>1200800", we certainly have missed some data, because reading is starting too late.
In that case, your image is incomplete, and gives misplaced blocks you are talking about.
This is why i try to find good values of parameters : latency, baudrates and timeouts.

To investigate, we have a simple solution i've already done for cam84 : switch back to D2XX library. i will try to do that this week end.

Your point about CPU usage is really interesting, i didn't noticed that.
For sure it is the reading process itself
On my Odroid XU4, reading the image takes a few seconds (<5/6s)

I had no time to try my raspberry 3, too many things at the same time (wife's car KO...)

Gilles.
Reply With Quote
  #507  
Old 15-01-2017, 01:19 AM
luka's Avatar
luka
Unregistered User

luka is offline
 
Join Date: Apr 2007
Location: Perth, Australia
Posts: 1,124
Thank you for the clarification, I understand much more now.

See the attached image, taken with the BBB and hand-held lens in front of Cam86.
One thing I am unsure about is how some "lines" do not shift the image but some have very drastic effect (see how the whole blocks of the image have been shifted around). I have a feeling there are two types of artifacts happening here, data loss (image shift) and also data mangling (visible lines but no image shift).

Also note black line on the bottom showing the missed data.

BBB is slow so all effects get emphasised. It may be useful to use a slow device for debugging the problem. I have noticed the "blocks" also on my PC but they don't happen very often.

And the windows driver also has lines but only on some machines. The issue probably exists everywhere but some machines handle the data transfer better and the issues are less obvious. My guess is that if enough images are taken on a machine that apparently works, the lines will eventually show.

I am just curious how this was not reported before on the Ukrainian forums, apart from Pat?
Attached Thumbnails
Click for full-size image (Light_001_e.jpg)
90.5 KB32 views
Reply With Quote
  #508  
Old 15-01-2017, 02:21 AM
luka's Avatar
luka
Unregistered User

luka is offline
 
Join Date: Apr 2007
Location: Perth, Australia
Posts: 1,124
Some testing on Windows. I recompiled the Ascom driver with latest version of FTD2XX_NET.DLL. No other changes. Tried it out using cam86_view on my laptop that usually shows lines and it did not make a difference, the problem was still there.

However, after about 10 images I got one that not only had lines but also blocks, just like we saw with your driver Gilles. Also for this particular problematic image cam86_view reported reading time of 3.7 seconds instead of the usual 2 seconds.
Reply With Quote
  #509  
Old 15-01-2017, 07:57 AM
rcheshire's Avatar
rcheshire (Rowland)
Registered User

rcheshire is offline
 
Join Date: Apr 2010
Location: Geelong
Posts: 2,624
Are some of those lines due to switching events? You guys are the electronic experts, but this problem seems almost universal. All my DSLR cooling mods required separating the analog - digital grounds and connecting the cold finger and camera chassis to power supply ground - no more lines.

Looking at the schematic and PCB, an inductor between mosfet source and ground, as well, might help.

Last edited by rcheshire; 15-01-2017 at 07:33 PM.
Reply With Quote
  #510  
Old 15-01-2017, 11:59 PM
gehelem (Gilles)
Registered User

gehelem is offline
 
Join Date: Sep 2016
Location: Rambouillet - France
Posts: 112
Quote:
Originally Posted by rcheshire View Post
Are some of those lines due to switching events? You guys are the electronic experts, but this problem seems almost universal. All my DSLR cooling mods required separating the analog - digital grounds and connecting the cold finger and camera chassis to power supply ground - no more lines.

Looking at the schematic and PCB, an inductor between mosfet source and ground, as well, might help.
Thank you for suggestion
I don't know, it might have some influence, but we have these lines even when cooling system is not plugged (ie no TEC on board)
We are searching on hardware/PC side, since some configuration seems to give more bugs than other (old laptop vs brand new vs arm ...)
Reply With Quote
  #511  
Old 16-01-2017, 12:07 AM
gehelem (Gilles)
Registered User

gehelem is offline
 
Join Date: Sep 2016
Location: Rambouillet - France
Posts: 112
i've added possibility to compile linux driver against D2XX library.
Not committed on github as it gives even worse results

New trick noticed : i have no problems when i use 2x2 bining
Reply With Quote
  #512  
Old 16-01-2017, 12:11 AM
pat30 (Patrice)
Registered User

pat30 is offline
 
Join Date: Oct 2016
Location: France
Posts: 119
The camera works better with my notebook (Toshiba nb250) quite old, same software, even Windows ?????

The only thing noticed is that the reading time is longer when there are lines.
Reply With Quote
  #513  
Old 16-01-2017, 02:28 AM
luka's Avatar
luka
Unregistered User

luka is offline
 
Join Date: Apr 2007
Location: Perth, Australia
Posts: 1,124
Rowland, thanks for the tip. Any ideas are greatly appreciated as we seem to be stuck. Mosfet switching is not likely to be the cause as, at the moment, I am not using any cooling and mosfet is off.

Gilles, I rewrote the ftdi_read_data function to use timeouts instead of a fixed number of loops. Check out my pull request on github.
I have set the timeout to 3 seconds as normal image reading should take about 2 seconds. Anything much over 2 seconds is bad and indicates problems.

Note how no data arrives for about first 1000ms and then we usually get all of it. The image may still contain lines indicating data corruption but no data loss. But if data is missing (blocks in image) we will get timeout.

Also I read somewhere that latency of 1 should not be used as it can lead to data loss. I changed the default values to 2 but it did not make any difference.

Then, what is 1 second delay (usleep ( 1000*1000 )) in readframe() doing (line 349)? I could not see any difference when I removed it. It is quite a long delay.

I tried 2x2 binning and it took lots of attempts but I got one line (on PC). So the problem is still there but less data to transfer means less chance for the problem to show itself.
When doing binning I noticed message
grabimage width=3000 height=1000 BPP=16
If we are doing 2x2 binning, why is the width still 3000?


Anyway, the problem is evident on Windows and Linux. Linux may be caused by drivers or our bugs and is worse but Windows... don't know. I noticed the same as Pat on Windows, longer reading time when there are lines or blocks. It seems to be machine dependent. I have one old laptop which has lines in 50% of the images (on Windows). Good for testing :-)

I am more convinced that there is either a bug in the FTDI drivers or in the Cam86 software/firmware. I mean at least under Windows the cam should work properly. I may need to join the Ukrainian forums and ask the guys there. It is weird how nobody there complained about the lines but we all are...

Filip, did you see any image corruption on your Cam86?
Reply With Quote
  #514  
Old 16-01-2017, 02:51 AM
pat30 (Patrice)
Registered User

pat30 is offline
 
Join Date: Oct 2016
Location: France
Posts: 119
Luka, dont you sleep ??
I still asked the question, on the forum Ukrainian, Rome gave other tracks but I think not good, for me, there is a problem with the driver but you never know.
Do you have an oscilloscope?

Last edited by pat30; 16-01-2017 at 03:10 AM.
Reply With Quote
  #515  
Old 16-01-2017, 03:04 AM
luka's Avatar
luka
Unregistered User

luka is offline
 
Join Date: Apr 2007
Location: Perth, Australia
Posts: 1,124
Plenty of time to sleep when I die
It is "only" midnight here (in Perth), just going to bed now.

Yes, I do have an oscilloscope.
Reply With Quote
  #516  
Old 16-01-2017, 03:07 AM
pat30 (Patrice)
Registered User

pat30 is offline
 
Join Date: Oct 2016
Location: France
Posts: 119
Quote:
Originally Posted by luka View Post
Plenty of time to sleep when I die
It is "only" midnight here (in Perth), just going to bed now.

Yes, I do have an oscilloscope.
It is a good vision of life

I thought later.

View forum Ukrainian to test with oscillo.
Reply With Quote
  #517  
Old 16-01-2017, 05:49 AM
gehelem (Gilles)
Registered User

gehelem is offline
 
Join Date: Sep 2016
Location: Rambouillet - France
Posts: 112
Quote:
Originally Posted by luka View Post
(...)
Gilles, I rewrote the ftdi_read_data function (...)
If we are doing 2x2 binning, why is the width still 3000?
(...)
Thank you very much, i've merged your code (first time i do that !)
I've corrected the readframe function : i've removed the unnecessary buffer purge, and the delay (don't know why i left that here...)
I fact, this way i've no more "blocked image", very good point, thanks again.
(EDIT : Do you want to be mentionned in author file ?)

Also, when you start an exposure, the message displayed was width=6000 (and 3000 when binning 2x2) it was just because we are working on 16bits and frame buffer is twice larger...
Correction done : i've divided by 2 when message is displayed

I think it's time to try on other platforms.
Raspberry 3, here i come !

Last edited by gehelem; 16-01-2017 at 05:59 AM.
Reply With Quote
  #518  
Old 16-01-2017, 01:19 PM
rcheshire's Avatar
rcheshire (Rowland)
Registered User

rcheshire is offline
 
Join Date: Apr 2010
Location: Geelong
Posts: 2,624
Pat and Luka.

I'm out of my depth in that case. I recall laptop monitor interference years ago with USB connected to an Arduino board for serial output to the monitor, but cooling was running. Interestingly, I no longer see this, even with switching events in the images. Software related. I take the point.
Reply With Quote
  #519  
Old 17-01-2017, 02:32 AM
luka's Avatar
luka
Unregistered User

luka is offline
 
Join Date: Apr 2007
Location: Perth, Australia
Posts: 1,124
Rowland, as I said any suggestions are greatly appreciated, thank you. You are probably predicting the problems that we will soon have, once the cooling comes on

Pat, I had a look at your images at the Ukrainian website and mine are slightly different. My images show vertical noise and the StdDev is higher than yours, I believe within the expected range (see image 1). But the lines in images are common. I did not have time to check the 15V power yet but I have a feeling it is not a hardware issue (at least not for my cam86).

Now, I made some progress with my tests. On one of my laptops I took 1000 images and all were 100% line-free. Then while taking images I started copying large files over WiFi and bang... see the attached image 2.

The issue was easily reproducible every time I started copying files over WiFi. I think WiFi card inside the laptop is connected to the internal USB hub and the bandwidth is shared. Hence simultaneous copying and imaging puts load on the USB bus and we get corrupted images.

Of course if you are using a desktop PC there may be multiple USB controllers and connecting the camera to a different USB port may give different results.

Gilles, this was all on Windows. The latest code still gives lines/blocks here. I actually tried a different USB port and it is worse, much much worse
I started working on new code to use ftdi_readstream as it may improve the reading performance but it is not working (yet), I just get blank images. Not sure if I am using it correctly, there is not much documentation out there.
Happy to be listed as an author, thank you.

I will do more tests tomorrow, need to stop now. Need to fix the power supply for my cooled 450D... can't wait for this build to be fully working
Attached Thumbnails
Click for full-size image (working.jpg)
153.6 KB18 views
Click for full-size image (working_long_USB_loaded.jpg)
89.6 KB14 views
Reply With Quote
  #520  
Old 17-01-2017, 07:11 AM
flolic (Filip)
Registered User

flolic is offline
 
Join Date: Jul 2016
Location: Split, Croatia
Posts: 48
Quote:
Originally Posted by luka View Post
Filip, did you see any image corruption on your Cam86?

So far, no.
Reply With Quote
Reply

Bookmarks

Thread Tools
Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +10. The time is now 05:51 AM.

Powered by vBulletin Version 3.8.7 | Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.
Advertisement
Bintel
Advertisement
Astromechanics
Advertisement
Testar
Advertisement
Lunatico Astronomical
Advertisement
Astronomy and Electronics Centre
Advertisement