- Description:
-
GSoC History API. Including sqlite history adapter
+ Remaining Items
+ Need to finish testing remove functionality in
purplesqlitehistoryadapter
~ Need to store database in purple config directory ~ Need to store database in purple config directory + Query builder currently leaks memory, we need to figure out how to get sqlite to free it. Other things to consider:
- History adapter parse queries & pass tokens to the adapters
GSoC History API
Review Request #877 — Created Aug. 4, 2021 and submitted
Information
GSoC History API including sqlite history adapter
The History API has been created to drive all message handling in purple3. It will be used to update existing messages for edits, reactions, pinning, read/deliver receipts, etc. The API uses an adapter pattern, to abstract out backends, but provides a SQLite3 backend by default.
It also provides search capabilities using a custom query language that can easily be expanded over time. It will be use by both the end user to search messages and the frontends to implement features like a pinned messages button. A command line utility is also provided for searching outside of the program itself.
Remaining Items
These all will most likely be done by the Pidgin core team after GSoC when we figure out exactly how to solve them.
Need to store database in purple config directory
* Gary has spent some time looking at this and it looks like the purple-history cli will need to become a purple-ui to make this work write as in the future other adapters will be plugins.Other things to consider:
- For simplicity, the SqliteHistoryAdapter is parsing the query itself, but for consistency havingPurpleHistoryAdapter
parse the query and pass tokens to the subclass might be something we want to do.
Unit Tests
History Manager
History AdapterIntegration Tests
purplehistorycore created for integration tests.
PurpleSqliteHistoryAdapter functionality tested:
- Creates proper db schema
- Writes logs
- Reads logs
- Queries using query language
- Deletes using query language
Description | From | Last Updated |
---|---|---|
I have only read tests and integration so far, not implementation. |
QuLogic | |
Comment block looks like it's missing a space for alignment |
sorvival | |
whitespace issues. |
grim | |
whitespace issues |
grim | |
whitespace issues |
grim | |
Same thing about comment block missing an alignment space |
sorvival | |
indentation |
grim | |
you don't need the #gchar here. |
grim | |
I would say something like "runs @query against @adapter"> |
grim | |
Perhaps "A list of message that match @query." reads better. |
grim | |
Missing @query |
grim | |
"Tells @adapter to remove messages that match @query." |
grim | |
"If removing the messages was successful". I wonder if we should make this return an int of how many messages … |
grim | |
These could probably just read %s does not implement the ... function. |
grim | |
There should always be a newline after variable declarations. |
grim | |
wrong indentation |
grim | |
missing active_changed. |
grim | |
This should be something more like "Removes messages from the active #PurpleHistoryAdapter of @manager that match @query." |
grim | |
"removed from the active adapter of @manager." |
grim | |
The #PurpleConversation instance. is plenty. |
grim | |
"Writes @message to the active adapter of @manager." |
grim | |
"%TRUE if @message was successfully written, %FALSE otherwise." |
grim | |
these should be in the same order they are in the Class structure so SIG_ACTIVE_CHANGED should be first. |
grim | |
Not sure how we missed a property for active, but we should probably add it. |
grim | |
This should be moved to line 270 so we don't need to unreference it if we're returning early because it's … |
grim | |
{ should be on a new line. |
grim | |
No need for the cannot perform query, bits in these. Who ever displays the error is likely going to put … |
grim | |
No need for the cannot perform remove, bits in these. Who ever displays the error is likely going to put … |
grim | |
No need for the cannot write message, bits in these. Who ever displays the error is likely going to put … |
grim | |
SQLite is the official capitalization. |
grim | |
don't need the extra blank lines here. |
grim | |
{ should be on a new line as the function parameters were wrapped. |
grim | |
`/ be careful with TRUE as it'll delete everything if theres nothing else in the query. / |
grim | |
we need to g_list_free_full(ins, g_free); each of the 3 lists here. |
grim | |
Might be nice to point out that sqlite prepared statements counts it's parameters starting at 1. |
grim | |
wrong indentation |
grim | |
wrong indentation |
grim | |
{ should be on a new line. |
grim | |
{ should be on a new line. |
grim | |
The check from purple_sqlite_history_adapter_write about checking if priv->db == NULL should be in here too. |
grim | |
we need to finish these. |
grim | |
{ should be on a new line. |
grim | |
The check from purple_sqlite_history_adapter_write about checking if priv->db == NULL should be in here too. |
grim | |
TODO |
grim | |
{ should be on a new line. |
grim | |
return should have an empty line above it unless it's the only statement in a block. |
grim | |
unnecessary newline |
grim | |
{ should be on the previous line as it didn't wrap. |
grim | |
no need for this to be a variable, just put the string in the function call. |
grim | |
need to add purplesqlitehistoryadapter.c as well. |
grim | |
Should just use the same %s does not implement the ... function like purple_history_adapter_query does. |
grim | |
Should just use the same %s does not implement the ... function like purple_history_adapter_query does. |
grim | |
Should just use the same %s does not implement the ... function like purple_history_adapter_query does. |
grim | |
Should just use the same %s does not implement the ... function like purple_history_adapter_query does. |
grim | |
This list shouldn't be sorted. |
grim | |
This should be transfer full and it's returning a list of PurpleMessage's not PurpleHistoryAdapter's. |
grim | |
newly activated |
grim | |
s/from/for/ ? |
grim | |
#PurpleHistoryAdapter |
grim | |
PurpleHistoryAdapter |
grim | |
should be } else { |
grim | |
the { shouldn't be indented |
grim | |
{ should be on a new line when parameters wrap. |
grim | |
{ should be on a new line when parameters wrap. |
grim | |
{ should be on a new line when parameters wrap. |
grim | |
{ should be on a newline when parameters wrap. |
grim | |
use purple_strequal insteqad of strcmp and all four of these. |
grim | |
return's should be separated by a newline unless they are the only statement in a block. |
grim | |
Looks like mixed indentation here. |
grim | |
This should be indented to the level of what it is documenting. |
grim | |
Extra space after (? |
QuLogic | |
Clear at end of test. |
QuLogic | |
Clear at end of test. |
QuLogic | |
Clear at end of test. |
QuLogic | |
Clear at end of test. |
QuLogic | |
replace the query variable with "query" there's no need for this to be a variable. |
grim | |
Clear at end of test. |
QuLogic | |
Clear all these at end of test. |
QuLogic | |
too many newlines |
grim | |
you should be checking the return vaolue of purple_history_manager_remove as well. |
grim | |
you should be checking the return vaolue of purple_history_manager_write as well. |
grim | |
Fix inconsistent names, m->manager, e->error, etc. |
QuLogic | |
Ditto. |
QuLogic | |
This should be query, not test-adapter, I think. |
QuLogic | |
Ditto. |
QuLogic | |
Aren't these all the defaults anyway? |
QuLogic | |
Ditto. |
QuLogic | |
This should go up near GLib and such, not after all the prpl-specific checks. |
QuLogic | |
Why is this top-level and not in libpurple somewhere?? |
QuLogic | |
This should make the command itself exit with EXIT_FAILURE. |
QuLogic | |
message needs to be unref'd before the link is deleted. |
grim | |
"failed to remove" |
QuLogic | |
Should also exit with failure. |
QuLogic | |
I think you can add g_option_context_set_help_enabled on the option context. |
QuLogic | |
This check is redundant with the for loop condition, since you don't do anything different if argc <= 1. You … |
QuLogic | |
THe |
grim | |
should be wrapped. |
grim | |
should be #PurpleHistoryAdapter's |
grim | |
should be #PurpleHistoryAdapter's as #PurpleHistoryAdapters isn't a known symbol to gtk-doc. |
grim | |
Should be #PurpleMessage's |
grim | |
the signals name is registered not adapter-registered so this should be PurpleHistoryManager::registered: |
grim | |
the signals name is unregistered not adapter-unregistered so this should be PurpleHistoryManager::unregistered: |
grim | |
should be wrapped |
grim | |
parameters wrapped so { should be on a newline. |
grim | |
no need to mention what it subclasses PurpleHistoryAdapter |
grim | |
"the instance read and writes to" is redundant and should be removed. |
grim | |
"Creates a new #PurpleSqliteHistoryAdapter." |
grim | |
"(transfer full): The new #PurpleSqliteHistoryAdapter instance" |
grim | |
The parameter documentation shouldn't contain the description of the function. It just needs to include the information about the parameter. … |
grim | |
You need a description here that is separated by a newlines between the parameters and the Returns. Something like *@adapter: … |
grim | |
extra space after * |
grim | |
we should be checking if these are empty strings before adding them to the lists and incrementing the query_items counter. … |
grim | |
need to free the lists here too. |
grim | |
This should be wrapped... something like. g_set_error(error, PURPLE_HISTORY_ADAPTER_DOMAIN, 0, _("Error opening database in PurpleSqliteHistoryAdapter " "for file %s"), priv->filename); |
grim | |
We typically create another variable of the type so we don't need to typecast in the _get_instance_private call which allows … |
grim | |
We typically create another variable of the type so we don't need to typecast in the _get_instance_private call which allows … |
grim | |
with the previous comment, having a variable of type PurpleSqliteHistoryAdapter avoid this line going past the boundary. |
grim | |
We typically create another variable of the type so we don't need to typecast in the _get_instance_private call which allows … |
grim | |
again, use the variable that the previous comment will create to avoid having a line that goes passed the boundary. |
grim | |
We typically create another variable of the type so we don't need to typecast in the _get_instance_private call which allows … |
grim | |
these all should be line wrapped. |
grim | |
putting sqlite3_errmsg(priv->db)); on a new line will get this line under 80 chars. |
grim | |
You don't need the PURPLE_HISTORY_ADAPTER() here. |
grim | |
extra newline |
grim | |
Since TestPurpleHistoryAdapter->query returns a list with 1 item you need to free it after the check with g_list_free. |
grim | |
uncomment and fix typo. |
grim | |
looks like this doesn't need to be wrapped? |
grim | |
looks like this doesn't need to be wrapped? |
grim | |
should be wrapped when over 80 chars. |
grim | |
should be wrapped after 80 chars |
grim | |
you can drop the PURPLE_MESSAGE() |
grim | |
You can drop the PURPLE_CONVERSATION() |
grim | |
This should wrap at 80 chars |
grim | |
previous and current are PurpleHistoryAdapter's not PurpleHistoryManager's. |
grim | |
There should be a space between sqlite3_stmt and the * |
grim | |
should be gint |
grim | |
there should be a new line before return when it's not the only statement in a block. |
grim | |
these three g_list_free_full's can be deleted now as the lists are cleaned up in the loops above and we've transfered … |
grim | |
there should not be a space between * and prepared_statement. |
grim | |
Keep math last. |
QuLogic | |
Missing the PURPLE_IS_HISTORY_ADAPTER check. |
QuLogic | |
Missing the PURPlE_IS_HISTORY_ADAPTER check. |
QuLogic | |
Returns: |
QuLogic | |
Returns: |
QuLogic | |
Extra space. |
QuLogic | |
The old one is deactivated at this point; should it be re-activated? |
QuLogic | |
Should be g_set_error_literal |
QuLogic | |
Should be g_set_error_literal. |
QuLogic | |
Should this be checking conversation or message, or is that dependent on specific adapters? |
QuLogic | |
Should be g_set_error_literal. |
QuLogic | |
SQLite belongs in the title? |
QuLogic | |
SQLite |
QuLogic | |
is a class |
QuLogic | |
defines inherits? |
QuLogic | |
Does something actually derive from this? It should be final if not. |
QuLogic | |
error is leaked. |
QuLogic | |
error is already set by g_resource_lookup_data, and error_msg is empty? |
QuLogic | |
purple_strequal returns TRUE when they're equal; this should not be comparing == 0. |
QuLogic | |
This is just split[i][3]. |
QuLogic | |
split[i][5] |
QuLogic | |
split[i][0] |
QuLogic | |
Since these are all ORd together, it doesn't appear that there's any benefit to putting this in any 'correct' order, … |
QuLogic | |
Should be g_error_set_literal |
QuLogic | |
If query_items is 0, then none of these would have anything in them. |
QuLogic | |
It would be shorter to write this part of the query as converation_id IN (?, ?, ?, ...). |
QuLogic | |
The whitespace probably doesn't matter, but did you intend to put newlines after each type of qualifier? |
QuLogic | |
Same here with author IN (?) |
QuLogic | |
Should be g_set_error_literal. |
QuLogic | |
priv->filename == NULL, so no point in including it in the error message like this, instead of explicitly calling that … |
QuLogic | |
If running migrations fails, then don't you need to close the db handle? |
QuLogic | |
Should be g_set_error_literal. |
QuLogic | |
Instead of SELECT *, you probably should have selected these specific columns. Then you don't need to pick random indices … |
QuLogic | |
Should be g_set_error_literal. |
QuLogic | |
Should you not finalize the prepared statement on error? |
QuLogic | |
This is const. |
QuLogic | |
Should be g_set_error_literal. |
QuLogic | |
Why is this not using the existing message_id? |
QuLogic | |
Should you not finalize the prepared statement on error? |
QuLogic | |
This needs to clear the db handle (and maybe warn) if it wasn't correctly unregistered. |
QuLogic | |
You should pass an error to all these tests and g_assert_no_error. |
QuLogic | |
Query is transfer container, not transfer full. Also, the test adapter returns GINT_TO_POINTER(1), so should definitely not be freed. |
QuLogic | |
Extra line break. |
QuLogic | |
Should be TRUE now. |
QuLogic | |
This is unused? |
QuLogic | |
Should be TRUE now. |
QuLogic | |
You aren't checking the return value to exit with the failure return code. |
QuLogic | |
Clear error. |
QuLogic | |
This indent seems off. |
QuLogic | |
It seems that adding these annotations for the error is unnecessary since the .gir doesn't not record it directly as … |
QuLogic | |
Is this opening in the current directory? It should go to the user config dir. |
QuLogic | |
Can drop the cast. |
QuLogic | |
Should verify that old is not NULL/non-adapter. |
QuLogic | |
Why are there two step calls? |
QuLogic | |
Here are the other ones. |
QuLogic |
- Change Summary:
-
Addressed memory changes
Validated remove functionality is working - Description:
-
GSoC History API. Including sqlite history adapter
Remaining Items
~ Need to finish testing remove functionality in
purplesqlitehistoryadapter
~ Need to store database in purple config directory
- Need to store database in purple config directory - Query builder currently leaks memory, we need to figure out how to get sqlite to free it. Other things to consider:
- History adapter parse queries & pass tokens to the adapters - Testing Done:
-
History Manager unit tests created
History Adapter unit tests created Integration test for History Sqlite History Adapter. - Sent messages ensured they were written to database - Checked that the query functionality works + - Checked that the remove functionality works Used purple-history CLI to test query functionality - Commit:
-
38c7dc89e06e72212eefd590
-
-
-
-
-
-
-
"If removing the messages was successful".
I wonder if we should make this return an
int
of how many messages were removed. -
-
-
-
-
This should be something more like
"Removes messages from the active #PurpleHistoryAdapter of @manager that match @query."
-
-
-
-
-
these should be in the same order they are in the
Class
structure soSIG_ACTIVE_CHANGED
should be first. -
-
This should be moved to line 270 so we don't need to unreference it if we're returning early because it's the active adapter.
-
-
No need for the
cannot perform query,
bits in these. Who ever displays the error is likely going to put that message there. -
No need for the
cannot perform remove,
bits in these. Who ever displays the error is likely going to put that message there. -
No need for the
cannot write message,
bits in these. Who ever displays the error is likely going to put that message there. -
-
-
-
-
-
-
-
-
-
-
The check from
purple_sqlite_history_adapter_write
about checking ifpriv->db == NULL
should be in here too. -
-
-
The check from
purple_sqlite_history_adapter_write
about checking ifpriv->db == NULL
should be in here too. -
-
-
-
-
-
-
-
-
Should just use the same
%s does not implement the ... function
likepurple_history_adapter_query
does. -
Should just use the same
%s does not implement the ... function
likepurple_history_adapter_query
does. -
Should just use the same
%s does not implement the ... function
likepurple_history_adapter_query
does. -
Should just use the same
%s does not implement the ... function
likepurple_history_adapter_query
does. -
-
This should be
transfer full
and it's returning a list ofPurpleMessage
's notPurpleHistoryAdapter
's. -
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
the signals name is
registered
notadapter-registered
so this should bePurpleHistoryManager::registered:
-
the signals name is
unregistered
notadapter-unregistered
so this should bePurpleHistoryManager::unregistered:
-
-
-
-
-
-
-
The parameter documentation shouldn't contain the description of the function. It just needs to include the information about the parameter. In this case
The #PurpleSqliteHistoryAdapter instance
. -
You need a description here that is separated by a newlines between the parameters and the
Returns
.Something like
*@adapter: The #PurpleSqliteHistoryAdapter instance. * * Gets the filename of the sqlite database. * * Returns: The file name of the sqlite data.
-
-
we should be checking if these are empty strings before adding them to the lists and incrementing the
query_items
counter.something like
if(split[i]+3 == '\0') { continue; }
For each block here obviously adjusting the offset as necessary.
-
-
This should be wrapped... something like.
g_set_error(error, PURPLE_HISTORY_ADAPTER_DOMAIN, 0, _("Error opening database in PurpleSqliteHistoryAdapter " "for file %s"), priv->filename);
-
We typically create another variable of the type so we don't need to typecast in the
_get_instance_private
call which allows us to avoid a line wrap. -
We typically create another variable of the type so we don't need to typecast in the
_get_instance_private
call which allows us to avoid a line wrap. -
with the previous comment, having a variable of type
PurpleSqliteHistoryAdapter
avoid this line going past the boundary. -
We typically create another variable of the type so we don't need to typecast in the
_get_instance_private
call which allows us to avoid a line wrap. -
again, use the variable that the previous comment will create to avoid having a line that goes passed the boundary.
-
We typically create another variable of the type so we don't need to typecast in the
_get_instance_private
call which allows us to avoid a line wrap. -
-
-
-
-
Since
TestPurpleHistoryAdapter->query
returns a list with 1 item you need to free it after the check withg_list_free
. -
-
-
-
-
-
-
-
- Description:
-
GSoC History API. Including sqlite history adapter
Remaining Items
~ Need to store database in purple config directory
~ Need to store database in purple config directory
+ * Gary has spent some time looking at this and it looks like the purple-history cli will need to become a purple-ui to make this work write as in the future other adapters will be plugins. Other things to consider:
- History adapter parse queries & pass tokens to the adapters
- Bugs:
- Bugs:
- Change Summary:
-
Fixed style issues. Fixed unfreed references. Add in checks against empty strings for sqlite adapter. Fixed documentation.
- Commit:
-
f7011529e33650900a670431
- Description:
-
GSoC History API. Including sqlite history adapter
Remaining Items
+ These all will most likely be done by the Pidgin team after GSoC after we figure out exactly how to solve them.
+ Need to store database in purple config directory
* Gary has spent some time looking at this and it looks like the purple-history cli will need to become a purple-ui to make this work write as in the future other adapters will be plugins. Other things to consider:
~ - History adapter parse queries & pass tokens to the adapters ~ - For simplicity, the SqliteHistoryAdapter is parsing the query itself, but for consistency having PurpleHistoryAdapter
parse the query and pass tokens to the subclass might be something we want to do.
- Description:
-
GSoC History API. Including sqlite history adapter
Remaining Items
~ These all will most likely be done by the Pidgin team after GSoC after we figure out exactly how to solve them.
~ These all will most likely be done by the Pidgin core team after GSoC when we figure out exactly how to solve them.
Need to store database in purple config directory
* Gary has spent some time looking at this and it looks like the purple-history cli will need to become a purple-ui to make this work write as in the future other adapters will be plugins. Other things to consider:
- For simplicity, the SqliteHistoryAdapter is parsing the query itself, but for consistency having PurpleHistoryAdapter
parse the query and pass tokens to the subclass might be something we want to do.
- Description:
-
GSoC History API. Including sqlite history adapter
+ The History API has been created to drive all message handling in purple3. It will be used to update existing messages for edits, reactions, pinning, read/deliver receipts, etc. The API uses an adapter pattern, to abstract out backends, but provides a SQLite3 backend by default.
+ + It also provides search capabilities using a custom query language that can easily be expanded over time. It will be use by both the end user to search messages and the frontends to implement features like a pinned messages button. A command line utility is also provided for searching outside of the program itself.
+ Remaining Items
These all will most likely be done by the Pidgin core team after GSoC when we figure out exactly how to solve them.
Need to store database in purple config directory
* Gary has spent some time looking at this and it looks like the purple-history cli will need to become a purple-ui to make this work write as in the future other adapters will be plugins. Other things to consider:
- For simplicity, the SqliteHistoryAdapter is parsing the query itself, but for consistency having PurpleHistoryAdapter
parse the query and pass tokens to the subclass might be something we want to do.
- Change Summary:
-
Address another around of PR comments. Fixed compiler warnings.
- Testing Done:
-
~ History Manager unit tests created
~ History Adapter unit tests created ~ Integration test for History Sqlite History Adapter. ~ - Sent messages ensured they were written to database ~ - Checked that the query functionality works ~ - Checked that the remove functionality works ~ Used purple-history CLI to test query functionality ~ Unit Tests
~ ~ History Manager
~ History Adapter ~ ~ Integration Tests
~ + purplehistorycore created for integration tests.
+ PurpleSqliteHistoryAdapter functionality tested: + - Creates proper db schema + - Writes logs + - Reads logs + - Queries using query language + - Deletes using query language - Commit:
-
50900a6704319afb84a45d95
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
Since these are all
OR
d together, it doesn't appear that there's any benefit to putting this in any 'correct' order, so there's no need to reverse the lists. -
-
-
-
The whitespace probably doesn't matter, but did you intend to put newlines after each type of qualifier?
-
-
-
priv->filename == NULL
, so no point in including it in the error message like this, instead of explicitly calling that out. -
-
-
Instead of
SELECT *
, you probably should have selected these specific columns. Then you don't need to pick random indices like starting from 3 and skipping from 9 up to 13. -
-
-
-
-
-
-
-
-
Query is
transfer container
, nottransfer full
. Also, the test adapter returnsGINT_TO_POINTER(1)
, so should definitely not be freed. -
-
- Change Summary:
-
address review findings and fix the unit tests. I'm having some issues reviewing the diff, so I'm publishing blind hoping it looks alright..
- Commit:
-
0474948f24678f4a02577e99
- Change Summary:
-
addressed open issues and collapsed everyting into a single commit as reviewboard was angry with me after I amended a commit and reviewboard is going to collapse everything anyways.
- Commit:
-
92bca10a280c00e6f98b4243
- Description:
-
~ GSoC History API. Including sqlite history adapter
~ GSoC History API including sqlite history adapter
The History API has been created to drive all message handling in purple3. It will be used to update existing messages for edits, reactions, pinning, read/deliver receipts, etc. The API uses an adapter pattern, to abstract out backends, but provides a SQLite3 backend by default.
It also provides search capabilities using a custom query language that can easily be expanded over time. It will be use by both the end user to search messages and the frontends to implement features like a pinned messages button. A command line utility is also provided for searching outside of the program itself.
Remaining Items
These all will most likely be done by the Pidgin core team after GSoC when we figure out exactly how to solve them.
Need to store database in purple config directory
* Gary has spent some time looking at this and it looks like the purple-history cli will need to become a purple-ui to make this work write as in the future other adapters will be plugins. Other things to consider:
- For simplicity, the SqliteHistoryAdapter is parsing the query itself, but for consistency having PurpleHistoryAdapter
parse the query and pass tokens to the subclass might be something we want to do.