Duplicates fd_loire

In eda2.3/sql/import_FDLoire_dbeel.sql line 511

-- TODO deal with duplicates in ang_id the number of rows inserted above was -- 4549 -- Insert length

some fish are duplicates and have the same ang_id This comes from /eda_dbeel/eda2.3/sql/join_loirebretagne_rn.sql

line 253 SELECT count(*) AS count, ob_id FROM dbeel.view_electrofishing_loirebretagne_frozen GROUP BY ob_id ORDER BY count DESC;

count	ob_id
4	a11d8cd3-61d7-4d42-a02e-2bb3054f97cd
4	626082dc-ceab-4b0b-b4d1-855b9951f4c5
4	ad81e34b-8d18-4442-9142-1bb71e14f54f
4	97170e13-8c15-484a-96f5-271ad3146ee4
4	db5b2c33-5d25-4e23-8347-7c2bdcbc192f
4	6b182104-9f14-4ea1-ba7b-52e37c8cfddf
4	6b51067e-b513-4b52-96ad-eb2b32138b7c
4	6a2c993a-ed97-46cb-92a3-8d031cb8e11b
1	61131a3e-5482-4c85-a03b-0412f465923a
1	ab6e618d-4bce-47d4-b16d-a2a93058b3ea

While searching, it's often operations without surface. They are duplicated in dbeel.view_electrofishing_loirebretagne_frozen,

For instance

SELECT * FROM  dbeel.observation_places
             JOIN dbeel.electrofishing ON observation_places.op_id = electrofishing.ob_op_id
             --JOIN dbeel.data_provider ON data_provider.dp_id = electrofishing.ob_dp_id
             --JOIN france.join_loirebretagne_rn ON join_loirebretagne_rn.op_id= observation_places.op_id            
             WHERE observation_places.op_id='d03c45c2-9ae5-49aa-b0ae-57bbd5e11df3';

reports several lines with same op_id and starting dates. This should be fixed. When searching for the source of data in seems that some are entered in dbeel.electrofishing and some others are entered in france dbeel_electrofishing_aspe dbeel_electrofishing_fdloire dbeel_electrofishing_rsa

For dbeel, the name of the table should be added as 'source_table', this would make things easier if we complexify the DB structure.

In R Eda_Loire.Rmd

unique(dbeel@datadbeelob_id[duplicated(dbeel@datadbeelob_id)]) [1] "97170e13-8c15-484a-96f5-271ad3146ee4" [2] "6b182104-9f14-4ea1-ba7b-52e37c8cfddf" [3] "db5b2c33-5d25-4e23-8347-7c2bdcbc192f" [4] "ad81e34b-8d18-4442-9142-1bb71e14f54f" [5] "6b51067e-b513-4b52-96ad-eb2b32138b7c" [6] "626082dc-ceab-4b0b-b4d1-855b9951f4c5"