The search function (poll)

[DinoToyForum]

As the poll clearly shows, the search works for some people some of the time. I'm trying to work out why it doesn't work for all people all of the time. For you, it never works, I think the forum gods have something against you. I'm trying to get to the bottom of it.

My last post was a bit destructive. Sorry for that.
"It never works" isn't exactly true. For example when I go to my collection thread and search for "Giant Salamander" on "this topic" I get zero results. When I go to the collection board doing the same search on "this board" I get at least one link to my collection thread. That's strange.

Yeah, it definitely isn't working properly, we can say that with certainly. 

[DinoToyForum]

The "This Topic" search still isn't working properly and I'm running out of ideas to fix it!

The code is all fine (the fact that it works at all shows that) so I wonder if it might be some server or database firewall setting, or some browser setting, since the issue seems to be browser dependent.

For me, on a Safari browser the search "entire forum" option works 100% of the time, and search "this topic" works 90% of the time. On a Chrome browser the search "entire forum" option works 90% of the time, and search "this topic" works only about 5%.

I also noticed that if you search for a word that definitely isn't on the forum (e.g. asdflfshflasufhl), it takes you to a page with a red error message that says "Your search query did not return any matches", whereas if you search for a word that definitely IS on the forum (e.g. "Triceratops) but it 'finds no matches', the page it takes you to is different and says (not an error message) "Sorry, no matches were found". The URL is the same in both cases.

@need4swede any ideas?

[need4swede]

@need4swede any ideas?

Strange, I just played around with the site's search feature and didn't have any issues whatsoever (Entire forum & This Topic)

Chrome Version

Spoiler

Version 101.0.4951.64 (Official Build) (arm64)

[close]

If this is a new issue, I'm wondering if there are any conditions carried over from the previous version of the site. If you have access to SQL tables, I would have a look and see what sort of queries are returning those results. Might give you a little bit more insight into what's going on. Might be some old Keys in there with conflicting conditionals.

This also goes without saying, but those who are having issues with the site; try clearing your browser's cache / history. Or try accessing the site in a private / incognito window. This could very easily be an issue with how cookies are being handled - Chrome is no stranger to this issue. I would start at those two and take it from there.

[ceratopsian]

@need4swede and @dinotoyforum - I still have the same problem.  I never had a search issue using Chrome before the change.  I've cleared my cookies (but not history yet).  I've accessed in incognito window.  It's made no difference.  (To be clear, this is a search on "This Topic".)  I seem to have far less difficulty with the Entire Forum search - though I note that postsaurichian's entry in his own thread of the word Iguanodon doesn't show up in either type of search.  "This topic" search for Iguanodon still turns up no hits in his thread.  Though "entire forum" search turns up someone else's comment in his thread.

[DinoToyForum]

@need4swede any ideas?

Strange, I just played around with the site's search feature and didn't have any issues whatsoever (Entire forum & This Topic)

Chrome Version

Spoiler

Version 101.0.4951.64 (Official Build) (arm64)

[close]

If this is a new issue, I'm wondering if there are any conditions carried over from the previous version of the site. If you have access to SQL tables, I would have a look and see what sort of queries are returning those results. Might give you a little bit more insight into what's going on. Might be some old Keys in there with conflicting conditionals.

This also goes without saying, but those who are having issues with the site; try clearing your browser's cache / history. Or try accessing the site in a private / incognito window. This could very easily be an issue with how cookies are being handled - Chrome is no stranger to this issue. I would start at those two and take it from there.

Thanks for your thoughts. I do have access to the database tables. Is there any particular query you recommend I run? I tried "SELECT * FROM mysql. general_log;" but it didn't return any results. However, I'm not very experienced so I may be doing something wrong.

[need4swede]

@need4swede and @dinotoyforum - I still have the same problem.  I never had a search issue using Chrome before the change.  I've cleared my cookies (but not history yet).  I've accessed in incognito window.  It's made no difference.  (To be clear, this is a search on "This Topic".)  I seem to have far less difficulty with the Entire Forum search - though I note that postsaurichian's entry in his own thread of the word Iguanodon doesn't show up in either type of search.  "This topic" search for Iguanodon still turns up no hits in his thread.  Though "entire forum" search turns up someone else's comment in his thread.

Thanks for the feedback. Indeed, this case doesn't work for me either. I'm pretty sure it's a matter of SQL tables that are causing this.

@dinotoyforum

If you haven't already, check 'Sources/Search.php' and verify that you can create temporary tables. You may have already tried this before. The information I have is for an older version of SMF, so it might not be an exact match, but the issues are all the same.

Around lines #1351 - 1368, do you see something like this?

Spoiler

Code Select Expand

db_query("
              INSERT IGNORE INTO {$db_prefix}log_search_results
                  (ID_SEARCH, ID_TOPIC, relevance, ID_MSG, num_matches)
              SELECT
                  " . $_SESSION['search_cache']['ID_SEARCH'] . ",
                  t.ID_TOPIC,
                  1000 * (
                    $weight[frequency] / (t.numReplies + 1) +
                    $weight[age] * IF(t.ID_FIRST_MSG < $minMsg, 0, (t.ID_FIRST_MSG - $minMsg) / $recentMsg) +
                    $weight[length] * IF(t.numReplies < $humungousTopicPosts, t.numReplies / $humungousTopicPosts, 1) +
                    $weight[subject] +
                    $weight[sticky] * t.isSticky
                  ) / $weight_total AS relevance,
                  t.ID_FIRST_MSG,
                  1
              FROM ({$db_prefix}topics AS t, {$db_prefix}" . ($createTemporary ? 'tmp_' : '') . "log_search_topics AS lst)
              WHERE lst.ID_TOPIC = t.ID_TOPIC" . (empty($modSettings['search_max_results']) ? '' : "
              LIMIT " . ($modSettings['search_max_results'] - $_SESSION['search_cache']['num_results'])), __FILE__, __LINE__);


[close]

The part to pay attention to is this:

Spoiler

WHERE lst.ID_TOPIC = t.ID_TOPIC" . (empty($modSettings['search_max_results']) ? '' : "

[close]

What it really should read is this:

Spoiler

WHERE " . (!$createTemporary ? 'lst.ID_SEARCH = ' . $_SESSION['search_cache']['ID_SEARCH'] . ' AND ': '') . 'lst.ID_TOPIC = t.ID_TOPIC' . (empty($modSettings['search_max_results']) ? '' : "

[close]

So it would turn out looking something like this:

Spoiler

Code Select Expand

db_query("
              INSERT IGNORE INTO {$db_prefix}log_search_results
                  (ID_SEARCH, ID_TOPIC, relevance, ID_MSG, num_matches)
              SELECT
                  " . $_SESSION['search_cache']['ID_SEARCH'] . ",
                  t.ID_TOPIC,
                  1000 * (
                    $weight[frequency] / (t.numReplies + 1) +
                    $weight[age] * IF(t.ID_FIRST_MSG < $minMsg, 0, (t.ID_FIRST_MSG - $minMsg) / $recentMsg) +
                    $weight[length] * IF(t.numReplies < $humungousTopicPosts, t.numReplies / $humungousTopicPosts, 1) +
                    $weight[subject] +
                    $weight[sticky] * t.isSticky
                  ) / $weight_total AS relevance,
                  t.ID_FIRST_MSG,
                  1
              FROM ({$db_prefix}topics AS t, {$db_prefix}" . ($createTemporary ? 'tmp_' : '') . "log_search_topics AS lst)
              WHERE " . (!$createTemporary ? 'lst.ID_SEARCH = ' . $_SESSION['search_cache']['ID_SEARCH'] . ' AND ': '') . 'lst.ID_TOPIC = t.ID_TOPIC' . (empty($modSettings['search_max_results']) ? '' : "
              LIMIT " . ($modSettings['search_max_results'] - $_SESSION['search_cache']['num_results'])), __FILE__, __LINE__);


[close]

From what I understand, this issue has effected previous version before, and it's a problem that has not been properly fixed and probably crept back with the update. I would take a look and see if the issue is related.

I would also check the following:

1. Are we using FullText index for searches? (Search index -> FullText index)
2. Is "Force the use of a search index" enabled or disabled?
3. Is "match whole words only" enabled or disabled?
4. What's the max search results set to? (0 = unlimited)

[DinoToyForum]

Thank you so much for your help. Yeah, the search.php file for SMF 2.1.2 appears to have the correct code – it isn't exactly the same as what you suggest, but includes "createTemporary" so I think it can create temporary tables (I used the clean 'Large Upgrade' files downloaded directly from SMF here: https://download.simplemachines.org).

Edit: I think this is the correct bit of code in the current search.php?:

Code Select Expand

$main_query['left_join'][] = '{db_prefix}' . ($createTemporary ? 'tmp_' : '') . 'log_search_topics AS lst ON (' . ($createTemporary ? '' : 'lst.id_search = {int:id_search} AND ') . 'lst.id_topic = t.id_topic)';
if (!$createTemporary)
$main_query['parameters']['id_search'] = $_SESSION['search_cache']['id_search'];
}


I would also check the following:

1. Are we using FullText index for searches? (Search index -> FullText index)
2. Is "Force the use of a search index" enabled or disabled?
3. Is "match whole words only" enabled or disabled?
4. What's the max search results set to? (0 = unlimited)

1. Custom Index. But originally 'No index'. I chose Custom to try and resolve the problem because it says on that page "A fulltext index is a built-in index of MySQL and PostgreSQL. It's relatively compact (approximately the same size as the message table), but a lot of words aren't indexed and it can, in some search queries, turn out to be very slow. The custom index is often bigger (depending on your configuration it can be up to 3 times the size of the messages table) but it's performance is better than fulltext and relatively stable."
2. Enabled, although I tested both options and neither fixed the problem.
3. Enabled, although I tested both.
4. 1200. I think I played around with the settings for this as well to no avail.

One thing to note, the forum database is MariaDB not PostgreSQL. Could there be a compatibility issue?

[need4swede]

I don't think there would be any compatibility issues, as both share a database model (Relational DBMS), but I wouldn't know for certain. Pretty sure it's fine.

I'll have to take a look and see how the search actually functions. I'll setup a server with 2.1.2 and dig around, probably later this week.

Are you able to temporarily disable all mods to see if one of them is conflicting with the search?

[DinoToyForum]

I already tried disabling key mods one by one with no luck but I'll try disabling them all together for good measure.

The weirdest thing to me is that it works for some people some time. Surely if there was an error in the code it wouldn't work at all, so there must be some variable at play.

I really appreciate your input 

[DinoToyForum]

I just tried disabling all the mods (on the ATF, which is facing the same search issue), but the search topic option still didn't work (for me in Chrome).

[DinoToyForum]

I did everything I did to fix this but concluded that the search feature in the new forum software is fundamentally broken. The software developers have acknowledged it. So, the search function remains a bit hit and miss for now. Hopefully it will be fixed in the next software update, but that's out of my hands, so we just have to live with it for now.

[ceratopsian]

Noted and thanks for the update @dinotoyforum. My half way house at present is to search in the advanced search by username and whatever it is I'm searching for. By no means perfect but better than nothing.

[DinoToyForum]

Noted and thanks for the update @dinotoyforum. My half way house at present is to search in the advanced search by username and whatever it is I'm searching for. By no means perfect but better than nothing.

It's deeply frustrating that the search function has become unfit for purpose. An exhaustive and reliable search function is one of the aspects of forums that make us better than other social media, so to be missing it now is a great loss. It's almost more annoying that it's hit and miss, at least we'd know where we stood if it was completely broken. Hopefully a fix will come sooner rather than later.

[Halichoeres]

Noted and thanks for the update dinotoyforum. My half way house at present is to search in the advanced search by username and whatever it is I'm searching for. By no means perfect but better than nothing.

This is a pretty useful workaround, and I'm glad it exists. Hopefully Adam is right and we'll get a fix in future updates to the forum software.