Bug #5275
closedImproper treatment of unknown fields for Alinco DJ-G7
0%
Description
First reported in #4355 but this is a separate bug.
The memory map of a channel for the DJ-G7 has two unknown fields, one of 3 bytes and one of 12 bytes -- see the DJG7EG_MEM_FORMAT struct. They are currently just skipped. Apparently, an empty channel has each and every byte set to 0xff. If chirp is used to set a channel that was previously empty, the fields are filled with data entered by the user but the unknown fields remain 0xff since they are being skipped.
That appears to work just fine with the radio, but the official Alinco software doesn't like it when a non-empty channel has 0xff in those unknown fields. It produces an error and bars the user from reading the radio.
Steps to fix:
- Try to figure out what those fields are or at least find the content written for non-empty channels by the Alinco software- I believe this is 0x00.
- Modify the code to explicitly write 0xff and 0x00(?) respectively when writing empty and non-empty channels.
- Add a fix to the code where we change the content from 0xff to 0x00 if a channel is found to be non-empty.
The last step is to make such radios readable by the Alinco software again.
Files
Updated by Emiel v about 7 years ago
Dear Mathias, Thank you for this very quick analysis; it seems to be the perfect explanation!
Good luck with the next steps! 73, Emiel
Updated by Emiel v about 7 years ago
After using Chirp to delete the records made in Chirp (keeping the ones made in clone utility); the clone utility makes a perfect all channel read on the remaining records.
But using Chirp to delete the chirp records and shifting the clone utility records up in position does not work; probably the hidden fields are not shifted upwards as well.
Of course this confirms your idea!
Updated by Mathias Weyland about 7 years ago
Thanks for looking at this once more. I think I have a preliminary patch that addresses the problem you reported. I'd like to clarify two things if I may, though:
When you say "and shifting the clone utility records up in position", how are you performing the shifting operation? I can think of several ways to do it: (a) Use the "Delete... and shift block up" feature, (b) Use the "Delete... and shift all memory up" feature, (c) Just delete and then use the "move up" feature, (d) Just delete, then cut & paste. I'd like to check what happens exactly with and without the patch when doing what you did since it would make sense to have it covered by the patch, too.
This is a bit weird: When I program your image to my radio, the first bank (created with Alinco software) looks good on the radio but the only channel displayed in the second bank is 2-62 Sky Radio. The 10 other channels above 62 can not be selected. It's as if the memory position was empty but the radio does not allow storing something there. Very strange, and even more strange is that I could swear that I could select those 10 channels the first time I had flashed your image. Unfortunately I only noticed this after a bit of testing of the patch so I can't assess if this was somehow broken by the patch or if the origin of the fault is somewhere else. Hence I'd like to ask if you could check if all 11 channels of bank 2 are available in your HT in the main band (= top line). Curiously they are displayed fine in the sub band. If they are not available then it will appear as if the radio was kinda stuck on channel 62 because that's the only one of the bank that can be accessed.
If this is the case in your radio too, don't panic. I have a fix for this condition as well. At first I thought the issue was permanent because I couldn't cure it even with the Alinco software or by resetting the radio but it turns out that it can be fixed easily.
Regards and thanks, Matt
Updated by Emiel v about 7 years ago
1 It was "Delete... and shift all memory up"; after that the clone utility did still not function.
2 In the cloning utility "bank tab" I allocated 150 channels in the DJ-G7 to bank 0; All other banks remained on the default 100 setting except the last one (9) which was set to 50 to have the total number not exceeding 1000. When re-opening the clone utility later you have to do a bank read to let it remember the new bank allocation.
The Chirp created channels were stored in records 1-111 hence bank 0
The channels created with the cloning utility were stored in bank 1 records 52 till 62. The first one is "10m FM aanroep" and the last one "Sky Radio". In Chirp they show up as 202-212. (150+52=202)
My DJ-G7EG shows bank 0 records in main, and bank 1 records in sub (that is the preferred setting for me anyway)
I added a new copy of the image file
Thanks again for looking into it; hope it is fun for you as well!
Updated by Mathias Weyland about 7 years ago
Thanks for your clarifications. Regarding the "Delete and shift up" issue, this is kind of hard for me to test because as soon as I load the image in the patched version, the broken fields get fixed. Whether I delete stuff afterwards or not doesn't make any difference at that point. This implies that deleting channels works, so I guess we can call that one fixed...
Regarding the "bank" issue: Looking at bank 2 in the sub band works for me. Looking at bank 2 in the main band does not really work (only shows 62-Sky). Can you quickly check if that's the case for you too? That test should be done before playing around with the patch discussed below. I think that the Alinco Software gets confused when it is modifying records generated by chirp (probably those that were not fully deleted) and messes them up even further. But as I said I believe to have understood how this happens and what needs to be done to fix it but it's not yet in the patch discussed below because I'm trying to understand whether this is only happening in the patched version or whether this is a problem that was already there but has only been discovered now...
All that said I have put together a patch that is supposed to deal "correctly" with those fields, including the removal of a channel. It also cures an image subject to this bug when opened or loaded into chirp. It does not yet address the "bank 2 only displays Sky in the main band" issue. If you are running Linux it should be easy to apply the patch if you want to give it a try. Otherwise I have to get it into the nightly build and you can go from there.
And yeah of course this is fun for me :-). At one point I'm hoping to add the channel block editor to chirp too, along with some of the other features but so far it has not happened unfortunately.
Updated by Emiel v about 7 years ago
In the set mode tab of the clone utility I had set "bank no (Main)" to 0 and bank no (Sub) to 1.
The alnico did just that; showing bank 0 channels in main en bank 1 channels in sub.
I changed (Sub) to bank0; and after that sub showed (only) the bank 0 channels.
No Linux here... just Windows 10 (64 bit). Have fun!
Updated by Emiel v about 7 years ago
sorry it was the "common set" tab of the clone utility
Updated by Emiel v about 7 years ago
and yes main only displays sky radio after it was set to bank 1 !!
maybe because that was the only channel I ever entered by hand into the alnico?
Updated by Emiel v about 7 years ago
The sky radio channel seems not to receive in main
Pressing 2x "main" key now sets main to bank 0
Updated by Mathias Weyland about 7 years ago
OK I think I figured it out. The "bank" issue is actually almost unrelated to this. Here's what's going on: The channels are represented in the image in a particular way. Every channel memory segment starts with a byte that indicates if the channel is empty (0x00) or not. For "not empty", I set this to 0x02 because at the time I did the coding, I observed what the Alinco software was doing and it appeared to just write 0x02 in there. In the bank that doesn't display any channel in the main band except for Sky, the "hidden" channels have 0x01. The reason for this is very simple: They are outside of the frequency range that is covered by the main band. Only the sub band has wide range coverage and AM / WFM demod I think. The Alinco software uses 0x01 on such channels to tell the radio that those only work in the sub band. As a consequence they are hidden in the main band.
I'm sorry I didn't catch this faster, it makes total sense given the receive architecture of the radio. I only use mine inside the ham bands so I had never tried WFM and/or out of band stuff. I'll update the patch to set 0x01 instead of 0x02 just like the Alinco software in these cases. This doesn't kill any feature because listening to Sky on the main band when chirp sets 0x02 simply doesn't work, the squelch opens but there is no audio. The radio hardware lacks the capability to do this in the main band.
Once this is done I'll submit the patch, it will end up in the official code base and you will be able to grab a nightly build for Windows. This will cure your image. In this context, I noticed that the last three channels in your 0-bank (AID 2013 and the two last Terlet ones) are subject to what is outlined above -- they are out of band and/or WFM. IF this all works like I imagine, you will find that you can program those with chirp for the main band but not with the Alinco software, yet they won't work in the main band even when "cheating" with the current chirp version.
Updated by Emiel v about 7 years ago
Thanks that is a real good job!
On 2m and 70cm the extended range of the main band is 136-169.995 and 420-469.995.
This requires the blue wire mod (that I havent used) but maybe it is an idea to use those limits (+23cm range) to select between 0x01 and 0x02? That also covers the T model ranges.
Thanks again!
Updated by Emiel v about 7 years ago
It might even be: 136 MHz - 173.995 MHz and 400MHz - 469.995 MHz see:
https://groups.yahoo.com/neo/groups/AlincoDJ-G7/conversations/messages/260
Updated by Emiel v about 7 years ago
Some web search:
1 extended range mod seems is used frequently
2 one more reference starting 70cm already on 400 MHz http://www.worldwidedx.com/threads/alinco-dj-g7-mod.35508/
Best range for 0x02 seems to be: 136-173.995 or 400-469.995 or 1240.000-1299.995 MHz
Updated by Emiel v about 7 years ago
E. van Puffelen wrote:
Some web search:
1 extended range mod is used frequently
2 one more reference starting 70cm already on 400 MHz http://www.worldwidedx.com/threads/alinco-dj-g7-mod.35508/
Best range for 0x02 seems to be: 136-173.995 or 400-469.995 or 1240.000-1299.995 MHz
Updated by Mathias Weyland about 7 years ago
Hello. Thanks again for your contributions. I've added a new patch (v2) with the following changes:
Don't fix the image in get_memory(). This might be a bit of an unpopular change because it means that a user has to touch the channels to fix them manually. Such channels are detected and reported in the log though. I've discussed this with other developers and they have a point in that it would be unexpected for a read-operation to modify the image.¶
Hide channels from the main band that can only be received in the sub band.¶
Regarding the last point, I went with the ranges that you suggested. Here is the relevant code-bit:
def _get_empty_flag(self, freq):
if (freq >= 136000000 and freq < 174000000) or \
(freq >= 400000000 and freq < 470000000) or \
(freq >= 1240000000 and freq < 1300000000):
return 0x02
else:
return 0x01
I hope I got the range right. I checked with a service monitor and the radio is indeed capable to receive FM in the full range so it makes sense to allow all of it. However, I believe there's one more slight issue: I think this should be further restricted to FM and NFM modes since the radio has no hardware to receive AM and WFM in the main band. I think that it wouldn't make any sense to have the radio allow a state where simply no signal could possibly be received... I'll add that change in a v3 patch and then it can go to the overlords for review and hopefully inclusion in the code base.
Regards and thanks
Matt
Updated by Emiel v about 7 years ago
1 With Chirp copy and pasting channel information in a new file is easy; an advice for anybody having trouble with an old image? "Old" image files will not be around much when the improved version is used some time.
2 Looks good!
Updated by Mathias Weyland almost 7 years ago
Finally, here's v3 of the patch. It contains everything v2 had, plus the restriction to FM and NFM for the main band as the hardware is not capable of receiving those in main band. Sorry this took so long, I'm about to submit this to the mailing list for review and inclusion in the code base.
Updated by Emiel v almost 7 years ago
Thanks, I had a look at the code and it looks very good!!! Somehow I am no longer able to download the patches as a recognisable .py file... so I couldn't test (with present chirp). But I am very confident you did it 100% perfect!
So please include... and I will check after that.
Thanks for this good job Mathias!
Updated by Mathias Weyland almost 7 years ago
Emiel, the patch should now be part of the chirp LATEST, in case you still feel like testing this.
Regards, Matt
Updated by Emiel v over 6 years ago
Math, just tested Chirp 20180412 and it works perfect!
It took a little up and down sending between chirp and the clone utility through the Alinco, to get rid of the wrong coding in the old 2017 channel file.
Now it is perfect: main shows frequencies between the limits of the post above and sub shows all including (far) out of band and AM.
The combination of Chirp and the Clone utility works fine. Clone utility for bank and other settings and chirp for easy channel editing and copying. You can have both programs running; just do read/write commands using one program at a time.
Thanks for this really excellent job!
73, Emiel
Updated by Mathias Weyland over 6 years ago
- Status changed from New to Resolved
Emiel, thank you for your feedback!
For the record: It should be able to resolve all broken channels with just one read/write cycle, but you have to manually touch every channel that needs fixing. Chirp automatically identifies those channels though and writes them to the log. I'm closing this issue since it is resolved and successful implementation of the fix has been confirmed. Thanks again!
Matt
Updated by Bernhard Hailer almost 5 years ago
- Status changed from Resolved to Closed
- Target version set to chirp-legacy
- Chirp Version changed from 0.4.0 to daily