Commit 66e81836 authored by Christine Plumejeaud's avatar Christine Plumejeaud
Browse files

added list of ship_id or captain_id to captain or ship description, with min...

added list of ship_id or captain_id to captain or ship description, with min and max date and the number of documents where they are mentioned.
parent 381f6dc1
......@@ -1145,8 +1145,9 @@ def getShipsDescription():
lang=validateLang(lang) # "fr" or "en" valid only (defaults to "fr")
localizedDatabaseFieldNames = f"occurences_homeports_{lang} as occurences_homeports, occurences_flags_{lang} as occurences_flags"
query = f"""select ship_id, occurences_names, {localizedDatabaseFieldNames}, occurences_class, occurences_tonnageclass
from navigoviz.ship"""
query = f"""select ship_id, occurences_names, {localizedDatabaseFieldNames},
occurences_class, occurences_tonnageclass, captain_list, mindate, maxdate, nb_sourcedoc
from navigoviz.ship"""
## Warning, no comments have been set on ship table
......@@ -1174,12 +1175,13 @@ def getCaptainsDescription():
localizedDatabaseFieldNames = f"occurences_birthplaces, occurences_citizenships"
# f"occurences_birthplaces_{lang} as occurences_birthplaces, occurences_citizenships_{lang} as occurences_citizenships"
query = f"""select captain_id, occurences_names, {localizedDatabaseFieldNames}
from navigoviz.captain"""
query = f"""select captain_id, occurences_names, {localizedDatabaseFieldNames},
ship_list, mindate, maxdate, nb_sourcedoc
from navigoviz.captain"""
##Warning, no comments have been set on captain table
## Si jamais on demande shortenFields= true, ca va planter.
return retrieveDataAndFormatOutput(query, api='ships')
return retrieveDataAndFormatOutput(query, api='captains')
@app.route('/api/test/')
def getTest():
......
......@@ -517,46 +517,63 @@ http://localhost/api/sources/?srid=4326&params=uhgs_id,toponym,pointcall_year,nb
http://localhost/api/sources/?srid=4326&params=uhgs_id,toponym,pointcall_year,source_suite,nb_conges_inputdone,nb_conges_cr,point&order=toponym,pointcall_year,source_suite&format=csv
```
# /ships?
## /ships?
Return the ships'list that we can follow in travels
Will be extracted from postgres, schema navigoviz, table ship (see sql/portic_detailviz.sql),
but with a filter on the lang : only fr or en attributes
- lang : **fr** | en, language for flag(_standardized), homeport, and birthplace (later on)
http://127.0.0.1:80/api/ships/?format=csv&lang=fr
http://127.0.0.1:80/api/ships/?format=json&lang=en
Parameters :
- lang : **fr** | en, language for flag(_standardized), homeport
Test :
Each ship attribute (name, homeport, flag, tonnageclass, class) is associated with the frequency of occurence in source documents in a JSON array format :
[{"variant1 of the attribute" : frequency}, {"variant2 of the attribute" : frequency}].
- http://127.0.0.1:80/api/ships/?format=csv&lang=fr
- http://127.0.0.1:80/api/ships/?format=json&lang=en
For instance :
Content:
Each ship attribute (name, homeport, flag, tonnageclass, class) is associated with the frequency of occurence in source documents in a JSON array format :
[{"variant1 of the attribute" : frequency}, {"variant2 of the attribute" : frequency}].
For instance :
0010348N [{"Negociant" : 1},{"Négociant de Bruges" : 1}] [{"Bruges" : 1},{"Hambourg" : 1}] [{"Bruges" : 1},{"Hamburg" : 1}] [{"hambourgeois" : 1},{"Pays-Bas autrichien" : 1}] [{"Austrian Netherlands" : 1},{"Hamburguese" : 1}] [{"[101-200]" : 2}]
This ship having 0010348N as identifier appears in 2 documents, named "Negociant" (once) or "Négociant de Bruges" (once). The ship's homeport is said "Bruges" in one case, "Hambourg" in the other case, and the ship's tonnage is between "101-200" tons in boths cases.
[{"00007942" : 2}] 1787-06-08 1787-12-06 2
0011092N [{"Vierge du Rosaire" : 2},{"Notre Dame du Rosaire" : 1}] [{"Gênes" : 1}] [{"Genoa" : 1}] [{"savoyard" : 2},{"génois" : 1}] [{"Savoyard" : 2},{"Genoese" : 1}] [{"bateau" : 2},{"tartane" : 1}]
This ship having 0010348N as identifier appears in 2 documents, named "Negociant" (once) or "Négociant de Bruges" (once). The ship's homeport is said "Bruges" in one case, "Hambourg" in the other case, and the ship's tonnage is between "101-200" tons in boths cases.
The captain was identified as "00007942" in both documents, and the ship has been moving on between le 8th of June to the 6th of December 1787
0011092N [{"Vierge du Rosaire" : 2},{"Notre Dame du Rosaire" : 1}] [{"Gênes" : 1}] [{"Genoa" : 1}] [{"savoyard" : 2},{"génois" : 1}] [{"Savoyard" : 2},{"Genoese" : 1}] [{"bateau" : 2},{"tartane" : 1}] [{"00011519" : 3}] 1787-03-30 1787-10-23 3
This ship having 0011092N as identifier appears in 3 documents, named "Vierge du Rosaire" (twice) or "Notre Dame du Rosaire" (once). The ship's homeport is said "Gênes" in one case, "savoyard" in the other case, and the ship's tonnage is unknown but the ship's class is said to be "bateau" in two documents, "tartane" in the third document.
The captain was identified as "00011519" in three documents, and the ship has been moving on between le 30th of March to the 23th of October 1787
# /captains?
## /captains?
Return the captains'list that we can follow in travels
Will be extracted from postgres, schema navigoviz, table captain (see sql/portic_detailviz.sql),
but with a filter on the lang : only fr or en attributes
- lang : **fr** | en, language for flag(_standardized), homeport, and birthplace (later on)
http://127.0.0.1:80/api/ships/?format=csv&lang=fr
http://127.0.0.1:80/api/ships/?format=json&lang=en
Parameters :
- lang : **fr** | en, language for citizenship and birthplace (translations are not yet available on June 2022)
Test :
- http://127.0.0.1:80/api/ships/?format=csv&lang=fr
- http://127.0.0.1:80/api/ships/?format=json&lang=en
Content:
Each captain attribute (name, citizenship, birthplace) is associated with the frequency of occurence in source documents in a JSON array format :
[{"variant1 of the attribute" : frequency}, {"variant2 of the attribute" : frequency}].
Each captain attribute (name, citizenship, birthplace) is associated with the frequency of occurence in source documents in a JSON array format :
[{"variant1 of the attribute" : frequency}, {"variant2 of the attribute" : frequency}].
For instance :
For instance :
00014569 [{"Accamo, Jean Baptiste" : 18},{"Accamo,Jean Baptiste" : 1},{"Accame, Jean Baptiste" : 1},{"Accamé, Jean Baptiste" : 1},{"Accamo, Jean BAptiste" : 1}] [{"Génois" : 18},{"Genois" : 3}]
00014569 [{"Accamo, Jean Baptiste" : 18},{"Accamo,Jean Baptiste" : 1},{"Accame, Jean Baptiste" : 1},{"Accamé, Jean Baptiste" : 1},{"Accamo, Jean BAptiste" : 1}] [{"Génois" : 18},{"Genois" : 3}] [{"0004424N" : 21}] 1787-01-07 1789-12-30 21
This captain idendified by the number 00014569 has been found named "Accamo, Jean Baptiste" 18 times, but only once named "Accame, Jean Baptiste". In 18 documents, he was a Génois, in 3 he was a Genois. The citizenship attribute has not been yet standardized.
In the 21 documents, he was sailing on a ship identified as "0004424N", between the 7th January 1787 to the 30th December 1789.
\ No newline at end of file
......@@ -321,8 +321,80 @@ comment on column navigoviz.captain.occurences_names is 'JSON array of various n
comment on column navigoviz.captain.occurences_birthplaces is 'JSON array of various birthplace of the same captain through sources, with the frequecy of appearance in documents';
comment on column navigoviz.captain.occurences_citizenships is 'JSON array of various citizenship of the same captain through sources, with the frequecy of appearance in documents';
-- suite le 16 juin : ajout de ship_id, captain_id et min et max date pour Bernard
alter table navigoviz.ship add column captain_list json;
alter table navigoviz.ship add column mindate date;
alter table navigoviz.ship add column maxdate date;
alter table navigoviz.ship add column nb_sourcedoc int;
alter table navigoviz.captain add column ship_list json;
alter table navigoviz.captain add column mindate date;
alter table navigoviz.captain add column maxdate date;
alter table navigoviz.captain add column nb_sourcedoc int;
comment on column navigoviz.ship.captain_list is 'JSON array of various captains having sailed on the same ship through sources, with the frequecy of appearance in documents';
comment on column navigoviz.ship.mindate is 'First date of appearance of this ship in the sources';
comment on column navigoviz.ship.maxdate is 'Last date of appearance of this ship in the sources';
comment on column navigoviz.ship.nb_sourcedoc is 'Number of documents mentionning this ship in the sources';
comment on column navigoviz.captain.ship_list is 'JSON array of various ships on which this captain has been sailing through sources, with the frequecy of appearance in documents';
comment on column navigoviz.captain.mindate is 'First date of appearance of this captain in the sources';
comment on column navigoviz.captain.maxdate is 'Last date of appearance of this captain in the sources';
comment on column navigoviz.captain.nb_sourcedoc is 'Number of documents mentionning this captain in the sources';
update navigoviz.ship s set captain_list = k.captain_list
FROM
(SELECT t.ship_id, array_to_json(array_agg(json_build_object(t.captain_id ,t.occurences))) as captain_list
FROM(
Select ship_id, captain_id , count (distinct source_doc_id) as occurences
FROM navigoviz.pointcall p
WHERE p.ship_id in (select distinct ship_id from navigoviz.built_travels)
GROUP BY ship_id, captain_id
ORDER BY ship_id, occurences desc
) t
WHERE occurences > 0 and captain_id is not null
GROUP BY t.ship_id
) as k
WHERE s.ship_id = k.ship_id;
update navigoviz.ship s set mindate = k.mindate, maxdate=k.maxdate, nb_sourcedoc=k.nb
from (
Select ship_id, min(p.date_fixed) as mindate, max(p.date_fixed) as maxdate, count(distinct source_doc_id) as nb
FROM navigoviz.pointcall p
WHERE p.ship_id in (select distinct ship_id from navigoviz.built_travels)
GROUP BY ship_id
) as k
WHERE s.ship_id = k.ship_id;
update navigoviz.captain s set ship_list = k.ship_list
FROM
(SELECT t.captain_id, array_to_json(array_agg(json_build_object(t.ship_id ,t.occurences))) as ship_list
FROM(
Select captain_id, ship_id , count (distinct source_doc_id) as occurences
FROM navigoviz.pointcall p
WHERE p.captain_id in (select distinct captain_id from navigoviz.built_travels)
GROUP BY captain_id, ship_id
ORDER BY captain_id, occurences desc
) t
WHERE occurences > 0 and ship_id is not null
GROUP BY t.captain_id
) as k
WHERE s.captain_id = k.captain_id;
update navigoviz.captain s set mindate = k.mindate, maxdate=k.maxdate, nb_sourcedoc=k.nb
from (
Select captain_id, min(p.date_fixed) as mindate, max(p.date_fixed) as maxdate, count(distinct source_doc_id) as nb
FROM navigoviz.pointcall p
WHERE p.captain_id in (select distinct captain_id from navigoviz.built_travels)
GROUP BY captain_id
) as k
WHERE s.captain_id = k.captain_id;
select * from navigoviz.ship where ship_id in ('0010348N', '0011092N')
select * from navigoviz.captain where captain_id in ('00014569')
-----------------------------------------------------------------------------------------
-- Reprise du travail sur maritime_paths_21fev2022
---------------------------------------------------------------------------------------------
......@@ -334,8 +406,9 @@ comment on column navigoviz.captain.occurences_citizenships is 'JSON array of va
-- has_input_done : vrai si il y a des sources pour ce port dans le G5 (faux à l'étranger toujours, et vrai pour Marseille)
/* rajouter ces attributs dans les commentaires des tables pour l'API pointcall */
-- fait sur le serveur le 15 juin 2022
-- l'algo fabrique public.test_pointcall automatiquement à partir de navigoviz.pointcall si on ne spécifie pas de filter
-- 2. L'algo va créer public.pointcall_verifs
-- 2. L'algo va créer public.pointcall_verifs à partir de public.test_pointcall qui est fabriqué à partir navigoviz.pointcall
-- 3. On demande ensuite la création de navigoviz.pointcall_checked (public.pointcall_heloise) sont la colonne net_route_marker est corrigée
-- et qui surtout qualifie le niveau de certitude qu'on a qu'un pointcall ait été fréquenté ou pas
-- 4. On corrige la fabrique de built_travels dans l'ETL (heloise_historiens) par autojointure sur navigoviz.pointcall_checked
......
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment