Project

General

Profile

Actions

Feature #10269

open

chirpc: add import and export CSV features

Added by Stuart Longland almost 2 years ago. Updated over 1 year ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
Start date:
01/14/2023
Due date:
% Done:

0%

Estimated time:
Chirp Version:
next (py3)
Model affected:
All
I read the instructions above:

Description

One thing I missed was the ability to just "import" a CSV file of channels that I had prepared earlier. This can be done in the GUI using copy and paste, but if that's not usable (e.g. if buggy libs cause a crash or if screen readers are in use), it'd be nice for chirpc to support this as it'd allow the memory channels to at least be manipulated without the full GUI.

I had a stab at this this morning and came up with this:
https://github.com/sjlongland/chirp/tree/feature/chirpc-csv-import-export

Usage is pretty simple:

  • Dumping: python3 chirpc --mmap radio.img --export-csv radio.csv
  • Loading: python3 chirpc --mmap radio.img --import-csv radio.csv

I've tried to stay "true" to the standard CSV format as much as possible. Export simply uses the Memory.to_csv method and the standard Memory headers. Import tries to be fairly permissive in terms of handling column ordering and extra columns. I did try using the Memory.really_from_csv method, but found the column indices it expects differ from the standard export format (notably; Mode was sourced from the RxCtcsCode column), so wound up writing my own.

I used it successfully to import channels for VK4 and some VK2 repeaters into my Yaesu FT-897D radio as generated by this script (which I adapted to support this CSV schema):
https://gist.github.com/sjlongland/16fa7308c8a6f5597e725e0b0d9f27f6

I suspect this code will need some cleaning up before final submission into CHIRP.

Actions #1

Updated by Dan Smith almost 2 years ago

Okay, but you should be able to just use the CSVRadio to read the CSV file and copy memories between that and the target radio. That will read the file header to get the columns, so that the user can reorder them and we'll still get them right, without using hard-coded indexes.

Also, the import_logic module provides all the glue that adapts one radio to another, because they have lots of different quirks. So when you copy/paste in the GUI, it uses import_logic to convert the memory from the target radio to the destination one.

Also, I have this queued up:

https://github.com/kk7ds/chirp/pull/391

I'm going to wait until tomorrow to merge it, but that will allow us to ship chirpc in the package and also break it apart and test it. I wouldn't have done that if I knew you were working on this, but it should be easy to rebase one or the other.

That all said, definitely cool to have native csv import in chirpc for sure.

Actions #2

Updated by Dan Smith almost 2 years ago

Something like this untested code:

out = the_radio
in = generic_csv.CSVRadio(inputfile)
features = in.get_features()
for i in range(*features.memory_bounds):
    m = in.get_memory(i)
    m = import_logic.import_mem(the_radio, features, m)
    the_radio.set_memory(m)

Should be all you need to read csv exactly like the GUI, as well as have the same import behavior, like converting memory names that won't fit in the target radio, etc. Also import_mem() will raise an error if a memory won't fit, like it is a band that the target radio doesn't support, etc.

Actions #3

Updated by Stuart Longland almost 2 years ago

Heh, I knew there must've been a cleaner way to do it, hadn't dug that deep into the code just yet.

I'll have a look at that and re-do the import code using CSVRadio when I get the chance.

Actions #4

Updated by Stuart Longland almost 2 years ago

BTW: don't let this hold up what you're doing… re-basing isn't a big issue and this might be a while. Packaging chirpc and testing it is certainly worth doing.

Actions #5

Updated by Fernando Bertuccelli over 1 year ago

The CSV export and import function is very useful for transferring memories from one radio model to another.
Without that, I don't really know how to do it.

Actions #6

Updated by Gerard Meszaros over 1 year ago

I depend on the Import feature because I have a much larger number of frequencies than memory slots on my radio. So I manage the frequencies in an Excel spreadsheet where I also keep information about each frequency that isn't downloaded to the radio (things like "Owner, Contact info, Last verified, Geographic Area, Type of Owner (e.g. Helicopter, Lodge, Park, etc. ) plus columns I use to filter the channels to include. Once I have all the frequencies I want loaded into the radio selected, I run a macro that copies them to a CSV file (with all essential columns plus extra columns to be ignored) and I import that into Chirp.

One thing I have a problem doing in "old" Chirp is getting it to ignore the old location column. I want it to import starting a 0 and number channels consecutively (because I order the memories in the spreadsheet.) Currently, I have to wipe out the contents of the Location column but still have the column header. (Chirp complains if there is no Location column. It shouldn't. No location column means "Import them in the order they appear in.")

Actions #7

Updated by Dan Smith over 1 year ago

Gerard and Fernando, I hope it's clear that this feature request is for chirpc, which is the command-line interface to chirp, not the UI. This request just represents adding those features to CLI only.

Actions #8

Updated by Gerard Meszaros over 1 year ago

OK, how do I put in a "missing feature" request for the UI? Can do it in Chirp-Old but not in Chirp-Next. I consider that a bug.

Actions #9

Updated by Jonathan Hill over 1 year ago

How have you managed to get on with this one? I'm currently considering using chirp for a number of big events we have, on the whole, we tend to have around 3 makes/models of handheld come in with players who need coding up(in the region of 60-200 players depending on site/time of year). it would be very useful to have for example a Raspberry pi running Chirpc able to encode the device, controlled by a GPIO button (I know Python can do this)

Was thinking it would even mean I could leave a Rpi in the bottom of the kit bag, in case I get to site and wish to re-code to my normal frequencies (ones supplied to my organisations commercial license.)

Actions #10

Updated by Doug Nelson over 1 year ago

Also having trouble with .csv import/export.

Actions

Also available in: Atom PDF